Python 开发包学习笔记
2019-07-25
- 使用 RMarkdown 的
child
参数,进行文档拼接。 - 这样拼接以后的笔记方便复习。
- 相关问题提交到 GitHub
You’ll learn the basics of creating Python packages that provide importable modules. Finally, you’ll learn how to write Conda recipes for your packages, build them, and share them on Anaconda Cloud. (DeFusco 2019)
参考 DeFusco (2019) 学习使用 conda 写包、建包和发布。
library(tidyverse)
1 Write Package
Turn your Python scripts into importable modules, collect those modules into packages and create Conda packages that can be easily installed by other users.
The first place Python will look for a module file is the current working directory. This means that any .py file you write can be used by other
.py
files in the same directory.
和 R 一样,Python 的建包过程也是把自己自定义的函数进行分享。但是和 R 不同,Python 更简单,每个 .py
就是一个 module,很方便理解。
In this chapter you’ll learn how to turn your Python scripts into importable modules, collect those modules into packages and create Conda packages that can be easily installed by other users. You now know that any
.py
Python file can act as a module. A module is a Python file that can be imported into a Python terminal or script file. The first place Python will look for a module file is the current working directory. This means that any.py
file you write can be used by other .py files in the same directory.
要注意路径,如果是文件夹,需要设定 _init_.py
.
The single most important aspect of developing a Python module is to separate importable expressions from execution statements. When a user calls import on your module they generally want to use your functions rather than wait for lengthy computation or watch many rows of output be printed.
引入但是不执行,很好的设计。 否则每次都需要先执行自定义函数,会比较浪费时间。
1.1 结构
Separating declarations from execution is the first step to make a module.
Here’s an example directory structure for my_package. Note the repeated use of my_package directories. The individual modules are stored in the inner my_package directory.
这是 Python 包的一种风格。
类似于 R 包,metadata .Rproj
名字和包名一致。
这是一个结构
my_package/
LICENSE
README
setup.py
my_package/
__init__.py
module1.py
module2.py
...
The setup.py script file coordinates installing the package into your Python distribution.
果然比 R 包更清晰。
(forecast) $ cat mortgage_forecasts/models.py
'''Models for predicting 30-year mortgage rates.'''
import warnings
warnings.filterwarnings("ignore")
import statsmodels.api as sm
import pandas as pd
from .utils import compute_margin
class MortgageRateModel(object):
'''A 12-order autoregressive model to predict mortgage rates.
Paramters
---------
mortgage_rates: Pandas Series with DateTimeIndex
The monthly mortgage rate to fit
a 12-order autoregressive model.'''
def __init__(self, mortgage_rates):
'''Initialize and fit the model.'''
self.mortgage_rates = mortgage_rates
self._model = sm.tsa.SARIMAX(mortgage_rates, order=(12, 0,0))
self.fit()
def fit(self, *args, **kwargs):
'''Fit the model
Parameters
----------
All parameters are passed to the statsmodels
SARIMAX fit() member method.
If disp is not passed it is set to False.'''
if 'disp' not in kwargs:
kwargs['disp'] = False
self.model = self._model.fit(*args, **kwargs)
def forecast(self, month, confint=0.95):
'''Forecast the 30-year mortgage rate over the given month.
Parmaters
---------
month: str
Parsible month-year to forecast
confint: float (<= 1)
Confidence interval to compute
margin of error
Returns
-------
rate : The forecasted mortgage rate
margin: The margin of error'''
last_known = self.mortgage_rates.index[-1]
if pd.to_datetime(month) < pd.to_datetime(last_known):
raise ValueError('The month to predict must be after {}' \
.format(last_known.strftime('%B %Y')))
forecast = self.model.get_prediction(last_known, month, confint=confint)
rate = forecast.predicted_mean.values[-1]
margin = compute_margin(forecast.se_mean.values[-1], confint)
return rate, margin
(forecast) $ cat mortgage_forecasts/utils.py
'''Auxilary tools for working with rate models.'''
import pandas as pd
from scipy.stats import norm
def compute_margin(stderr, confint=0.95):
'''Compute margin of error from standard error
Paramters
---------
stderr: The standard error of a prediction
confint: The desired confidence interval
(defaults to 0.95)
Returns
-------
margin_of_error: fractional margin of error'''
z = norm.ppf((1+confint)/2)
return stderr * z
def read_data(filename):
'''Read mortgage rate data
Parameters
----------
filename: path to CSV file
Formatted as https://fred.stlouisfed.org/series/MORTGAGE30US
Returns
-------
rates: Pandas Series with datetime index'''
df = pd.read_csv(filename, index_col='DATE', parse_dates=True)
rates = df['MORTGAGE30US']
monthly = rates.resample('MS').first()
return monthly
$ cat mortgage_forecasts/__init__.py
'''Predictive modeling of 30-year mortgage rates in the US.'''
from .models import MortgageRateModel
from .utils import read_data
.models
表示在当前路径的models
文件或者文件夹引入函数。
1.2 setup.py
最简单的 metadata 写法
$ cat setup.py
from setuptools import setup, find_packages
setup( packages=find_packages(),
name = "mortgage_forecasts",
author = "Me",
description = "30 year mortgage rate models",
)
类似于 建立 R 包时的 DESC 文档。
1.2.1 License
When someone else downloads and uses the program we built we would not want to transfer ownership to them, thereby forfeiting our rights. Further, license=“MIT” has been added in setup.py.
这句话说得好,我也要注意。
The MIT license does
notallow commercial use.
1.2.2 Version
$ cat __init__.py
'''Predictive modeling of 30-year mortgage rates in the US.'''
from .models import MortgageRateModel
from .utils import read_data
__version__ = '0.1'
(forecast) $ cat setup.py
from setuptools import setup, find_packages
from version import find_version
setup(
name = 'mortgage_forecasts',
author = 'Me',
description = '30-year mortgage rate models',
license = 'MIT',
version = find_version('mortgage_forecasts', '__init__.py'),
packages = find_packages()
)
1.2.3 Packages
The final step is to specify what directories setuptools should use to find the package source files.
The find_packages() function searches the directory where setup.py resides and returns a list of all subdirectories that have an init.py file.
Add packages = find_packages() as an argument to setup.py.
find_packages
就是为了安装 dependency。
要测试的时候,别忘了先。
python setup.py install
这是一个很常见的命令行,常见于 GitHub 上面挂的 Python 包,在需要安装时,先安装 Dependency。
2 Build Package
In this chapter you’ll create a Conda Recipe for the mortgage_forecasts package to define the dependent Conda packages. The Conda recipe is specified in a file called meta.yaml.
You’ll then build the package archive and upload it to Anaconda Cloud.
之前做的是完成包必要的一部分代码,下面类似于 docker 和 binder ,把配置信息进入录入。
$ conda install conda-build
类似于 R 的 devtools
Why are there build: and run: sections in requirements:?
类似于 R 包内的 Depend 和 Suggest,但是不尽相同。
The glob * means that the preceding characters must match exactly.
requirements:
run:
- python
- scipy
- numpy 1.11*
- matplotlib >=1.5, <2.0
Any version of NumPy after 1.11.x or before 1.12.0
*
表示安装对应 1.11 限定的版本。
If the script: tag is not present in the meta.yaml file then conda build will expect the installation commands to be in a file called build.sh on Mac and Linux, or in build.bat on Windows.
build:
script: python setup.py install --single-version-externally-managed --record=record.txt
1.2.3
python setup.py install
常用来安装 dependency,这里写入 yaml 文档后,就不需要 End Users 操作了。
End Users 不具备开发包的能力,这样更加用户友好。
conda install mortgage_forecasts pandas=0.19 --use-local
本地安装的方法。
noarch Conda package.
This means that only one package archive is required and can be installed with any version of Python on any architecture supported by Conda.
noarch: python
number: 1
$ cat meta.yaml
{% set setup_py = load_setup_py_data() %}
package:
name: 'mortgage_forecasts'
version: {{ setup_py.get('version') }}
source:
path: ./
build:
noarch: python
number: 1
script: python setup.py install --single-version-externally-managed --record=record.txt
requirements:
run:
- python >=2.7
- pandas >=0.20
- statsmodels
- scipy
build:
- python
- setuptools
about:
license: {{ setup_py.get('license') }}
license_file: LICENSE
summary: {{ setup_py.get('description') }}
$ anaconda upload noarch/mortgage_forecasts-0.1-py_1.tar.bz2
上传成功。
The next step is to add the build requirements. The packages listed under build: are only those required to run python setup.py. In our case only python and setuptools are required.
注意
build:
- python
- setuptools
这就是为什么 build 只需要两个包的原因。
conda build .
3 Destribute Package
3.1 Zenodo
如这个 zenodo 项目,可以作为一个开源文献,给 BibTex、参考文献、DOI。 Zenodo 帮助可以申请 DOI。 在 https://zenodo.org/account/settings/github/ 中进行关联。
create a github release
在 GitHub 建立一个 release 后,就会自动生成一个 zenodo。同时产生一个如图的 badge。
https://help.github.com/en/articles/creating-releases 查看如何构建 release。
在 R 中输入usethis::use_github_release()
也可。
> use_github_release()
✔ Setting active project to '/Users/vija/Downloads/work/add2bibtex'
✔ Checking that remote branch 'origin/master' has the changes in 'local/master'
✔ Opening URL 'https://github.com/JiaxiangBU/add2bibtex/releases/tag/untagged-832ee88b492d620dcad8'
Creates a draft GitHub release for the current package using the current version and NEWS.md. If you are comfortable that it is correct, you will need to publish the release from GitHub.
具体信息可以参考 https://usethis.r-lib.org/reference/use_github_release.html
发布后的情况可以参考 https://github.com/JiaxiangBU/add2bibtex/releases/tag/v0.5.1
生成的 BibTex https://zenodo.org/record/3345141/export/hx#.XTSjeJMzayA
@misc{jiaxiang_li_2019_3345141,
author = {Jiaxiang Li},
title = {JiaxiangBU/add2bibtex: add2bibtex 0.5.1},
month = jul,
year = 2019,
doi = {10.5281/zenodo.3345141},
url = {https://doi.org/10.5281/zenodo.3345141}
}
Jiaxiang Li. (2019, July 21). JiaxiangBU/add2bibtex: add2bibtex 0.5.1 (Version v0.5.1). Zenodo. http://doi.org/10.5281/zenodo.3345141
3.2 Pypi
We use gravatar.com to generate your profile picture based on your primary email address — alex.lijiaxiang@foxmail.com
使用 gravatar.com 进行头像管理。
用于生成 PyPi badge 的服务可以参考
主要教程参考官方 https://packaging.python.org/tutorials/packaging-projects/
注意目前托管的服务有两个一个是生产的 pypi.com 一个是测试的 test.pypi.com。 两边都需要注册账号。
(base-37) D:\work\pyks>python -m twine upload --repository-url https://test.pypi
.org/legacy/ dist/*
Enter your username: JiaxiangBU
Enter your password:
Uploading distributions to https://test.pypi.org/legacy/
Uploading pyks-0.1.1-py3-none-any.whl
90%|█████████████████████████████████▎ | 8.
100%|█████████████████████████████████████|
8.87k/8.87k [00:01<00:00, 6.16kB/s]
NOTE: Try --verbose to see response content.
HTTPError: 403 Client Error: Invalid or non-existent authentication information.
for url: https://test.pypi.org/legacy/
.HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/
出现这个报错一般是用户名和密码错误了,要去网站上验证一下你前面注册的用户名和密码。 https://www.cnblogs.com/zqifa/p/pypi-1.html
生成的网址就是 https://test.pypi.org/project/pyks
在测试成功后,再进行生产上的上传。
系统性的构建代码,参考
read_file("https://raw.githubusercontent.com/JiaxiangBU/pyks/master/R/dev_history.R") %>% cat
## file.copy("../mortgage_forecasts/LICENSE", ".")
## package_name <- "pyks"
## library(tidyverse)
## library(glue)
##
## # once --------------------------------------------------------------------
##
## file.copy("../mortgage_forecasts/setup.py", ".")
## file.copy("../mortgage_forecasts/version.py", ".")
## file.copy("../mortgage_forecasts/meta.yaml", ".")
## dir.create("pyks")
## file.copy("../mortgage_forecasts/mortgage_forecasts/__init__.py", "pyks/", overwrite = TRUE)
## file.copy("../mortgage_forecasts/mortgage_forecasts/models.py", "pyks/ks.py", overwrite = TRUE)
## file.edit("pyks/__init__.py")
##
## use_readme_rmd()
## file.copy("../mortgage_forecasts/README.Rmd", "README.Rmd", overwrite = TRUE)
##
## read_file("README.Rmd") %>% str_replace_all("mortgage_forecasts", package_name) %>% write_file("README.Rmd")
##
##
## # func writing ------------------------------------------------------------
##
## file.edit("pyks/ks.py")
##
##
## # ks v2 -------------------------------------------------------------------
##
## file.copy("pyks/ks.py", "pyks/ks2.py")
##
##
## # Add DESC ----------------------------------------------------------------
##
##
##
## desc_text <- read_lines("setup.py") %>%
## str_subset("description") %>%
## str_match("'([[A-z]\\s]+)'") %>%
## .[1,2]
## desc_file <- read_lines("DESCRIPTION")
## desc_file[desc_file %>% str_which("Title")] <- glue("Title: {desc_text}")
## desc_file[desc_file %>% str_which("Description")] <- glue("Description: {desc_text}")
## desc_file %>%
## write_lines("DESCRIPTION")
##
## add2pkg::add_me()
##
## options(usethis.full_name = "Jiaxiang Li")
## use_mit_license()
##
## # 所以还是要用 R Package 的框架
##
## use_news_md()
## file.edit("NEWS.md")
##
##
## # update version ----------------------------------------------------------
##
## file.edit("NEWS.md")
## library(devtools)
## use_version()
##
## version_text <- read_lines("DESCRIPTION") %>%
## str_subset("Version") %>%
## str_match("(\\d+\\.\\d+\\.\\d+)") %>%
## .[1,2]
## init_file <- read_lines("pyks/__init__.py")
## init_file[init_file %>% str_which("__version__")] <- glue("__version__ = '{version_text}'")
## init_file %>% write_lines("pyks/__init__.py")
##
##
##
##
## # release on Anaconda -----------------------------------------------------
##
## # conda build . # conda
##
## library(fs)
## conda_bld_path <- if(sessioninfo::os_name() %>% str_detect("Windows")) {
## "../../software/anaconda/envs/base-37/conda-bld/noarch/"
## } else {
## "/Users/vija/miniconda3/conda-bld/noarch/"
## }
## upload_file_path <-
## dir_info(conda_bld_path)$path %>%
## str_subset("pyks") %>%
## max
##
## # anaconda login
##
## glue("anaconda upload {upload_file_path}") %>% clipr::write_clip() %>% cat
##
##
## # release on PyPi ---------------------------------------------------------
##
##
## # Python Package Index
##
## # https://packaging.python.org/tutorials/packaging-projects/
## file.edit("setup.py")
## file.edit("pyks/__init__.py")
## # python -m pip install --user --upgrade setuptools wheel
## # Successfully installed wheel-0.33.4
## # python setup.py sdist bdist_wheel
##
## # $ ls dist
## # pyks-0.1.1-py3-none-any.whl pyks-0.1.1.tar.gz
##
## # python -m pip install --user --upgrade twine
## # python -m twine upload --repository-url https://test.pypi.org/legacy/ dist/* --verbose
## # python -m twine upload dist/*
##
##
## # release on GitHub -------------------------------------------------------
##
## usethis::use_github_release() # Use MacOS
3.3 常见问题
3.3.1 meta.yaml 申明正确的名字
In MacOS
conda build .
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Copying /Users/vija/Downloads/work/pyks to /Users/vija/miniconda3/conda-bld/pyks_1563990260616/work/
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for pyks
INFO:conda_build.metadata:Attempting to finalize metadata for pyks
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
BUILD START: ['pyks-0.1.1-py_1.tar.bz2']
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed
Leaving build/test directories:
Work:
/Users/vija/miniconda3/conda-bld/work
Test:
/Users/vija/miniconda3/conda-bld/test_tmp
Leaving build/test environments:
Test:
source activate /Users/vija/miniconda3/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_
Build:
source activate /Users/vija/miniconda3/conda-bld/_build_env
Traceback (most recent call last):
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 756, in get_install_actions
actions = install_actions(prefix, index, specs, force=True)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/plan.py", line 474, in install_actions
txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 112, in solve_for_transaction
force_remove, force_reinstall)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 150, in solve_for_diff
force_remove)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 245, in solve_final_state
ssc = self._add_specs(ssc)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 477, in _add_specs
explicit_pool = self._get_package_pool(ssc, self.specs_to_add)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 396, in _get_package_pool
pool = ssc.r.get_reduced_index(specs)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 485, in get_reduced_index
explicit_specs, features = self.verify_specs(explicit_specs)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 290, in verify_specs
raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound:
- sklearn
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/vija/miniconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 445, in main
execute(sys.argv[1:])
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 436, in execute
verify=args.verify, variants=args.variants)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 209, in build
notest=notest, need_source_download=need_source_download, variants=variants)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2313, in build_tree
notest=notest,
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1377, in build
create_build_envs(top_level_pkg, notest)
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1279, in create_build_envs
raise e
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1269, in create_build_envs
channel_urls=tuple(m.config.channel_urls))
File "/Users/vija/miniconda3/lib/python3.7/site-packages/conda_build/environ.py", line 758, in get_install_actions
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: {'sklearn'}
sklearn
名字错了。
3.3.2 版本显示
这是因为 那应该就是当初上版本时,写的 v1.0.0 的问题了,后来才弄成 v.0.1.0 开始的。 我相当于版本回滚了,修改回来,验证如下。
附录
3.4 在 shell 执行 Python
$ python pi.pypi is approximately 3.1415926
$ pythonPython 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pi
这是 Python interpreter 类似于 R 和 RStudio 和 shell 的关系。
注意在这里是在 shell 中执行 Python 命令的方式。
如果是脚本再使用python xxx.py
3.5 Anaconda Project
Anaconda Projects allow you to package code, data, and Conda environments for others to use easily. Starting with simple data science applications you’ll create Anaconda Project archives that enable reproducible data science.
除了托管到 GitHub,这里为了 End Users 方便安装也会考虑放到 Anaconda Cloud 上。
conda install anaconda-project
$ anaconda-project list-packages
Packages for environment 'default':
pandas
python=3
基于文件
$ ls
anaconda-project.yml main.py
在当前路径会生成一个新的环境
anaconda-project prepare
$ anaconda-project list-commands
Commands for project: /home/repl/babynames
Name Description
==== ===========
search-names python main.py
总体看挺方便的。
Initialize a new project
anaconda-project.yml
建立一个新的 Python 包
Now you have initialized the mortgage_rates project, it’s time to add packages, downloads, and commands.
anaconda-project add-packages python=3 pandas statsmodels
类似于 conda 新建包。
增加需要的 packages
我可以自定义我的
$ anaconda-project add-download MORTGAGE_RATES https://s3.amazonaws.com/assets.datacamp.com/production/course_6859/datasets/MORTGAGE30US.csv
数据源来自 Freddie Mac (2019)
$ cat forecast.py
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
import os
import pandas as pd
import statsmodels.api as sm
### Retrieve MORTGAGE_RATES from the environment
MORTGAGE_RATES = ...
df = pd.read_csv(MORTGAGE_RATES, index_col='DATE', parse_dates=True)
trimmed = df.loc['2010':].resample('MS').first()
model = sm.tsa.SARIMAX(trimmed['MORTGAGE30US'], order=(12, 0,0)).fit(disp=False)
YEAR = os.environ['YEAR']
forecast = model.get_prediction('{}-1-1'.format(YEAR), '{}-12-1'.format(YEAR), confint=0.95).summary_frame()
print('Forecasted {} 30-year mortgage rates at the 95% confidence
interval'.format(YEAR))
for date, row in forecast.iterrows():
month = date.strftime('%B')
rate = row['mean']
margin = row['mean_se'] * 1.96
result = '{:10s} {:.2f}% (+/- {:.3f})'.format(month, rate, margin)
print(result)
这是 forecast 脚本,对应的数据也要下载。
$ ls
MORTGAGE30US.csv anaconda-project.yml envs forecast.py
MORTGAGE_RATES = os.environ["MORTGAGE_RATES"]
anaconda-project add-command --type unix forecast "python forecast.py"
anaconda-project add-command [flags] <name> <command-to-execute>
The script file is now ready and the next step is to register it as a command in your project. The script file is intended to be run as a command-line tool and must be declared a Unix command with
--type unix
.
这是限定 UNIX 的原因。
$ anaconda-project run forecast
anaconda-project update
更新项目。
anaconda-project lock
锁定版本,会出现一个文件,执行这个代码还挺慢的。
$ cd mortgage_rates$ ls
anaconda-project.yml envs forecast.py$ anaconda-project lock
Updating locked dependencies for env spec default...
Resolving conda packages for linux-64
ls
Resolving conda packages for osx-64
Resolving conda packages for win-64
Changes to locked dependencies for default:
platforms:
+ linux-64
+ osx-64
+ win-64
packages:
+ all:
+ blas=1.0=mkl
+ ca-certificates=2019.5.15=0
+ certifi=2019.6.16=py37_0
+ patsy=0.5.1=py37_0
+ pip=19.1.1=py37_0
+ python-dateutil=2.8.0=py37_0
+ pytz=2019.1=py_0
+ setuptools=41.0.1=py37_0
+ six=1.12.0=py37_0
+ wheel=0.33.4=py37_0
+ linux-64:
+ _libgcc_mutex=0.1=main
+ intel-openmp=2019.4=243
+ libedit=3.1.20181209=hc058e9b_0
+ libffi=3.2.1=hd88cf55_4
+ libgcc-ng=9.1.0=hdf63c60_0
+ libgfortran-ng=7.3.0=hdf63c60_0
+ libstdcxx-ng=9.1.0=hdf63c60_0
+ mkl=2019.4=243
+ mkl_fft=1.0.12=py37ha843d7b_0
+ mkl_random=1.0.2=py37hd81dba3_0
+ ncurses=6.1=he6710b0_1
+ numpy-base=1.16.4=py37hde5b4d6_0
+ numpy=1.16.4=py37h7e9f1db_0
+ openssl=1.1.1c=h7b6447c_1
+ pandas=0.24.2=py37he6710b0_0
+ python=3.7.3=h0371630_0
+ readline=7.0=h7b6447c_5
+ scipy=1.2.1=py37h7c811a0_0
+ sqlite=3.28.0=h7b6447c_0
+ statsmodels=0.10.0=py37hdd07704_0
+ tk=8.6.8=hbc83047_0
+ xz=5.2.4=h14c3975_4
+ zlib=1.2.11=h7b6447c_3
+ osx-64:
+ intel-openmp=2019.4=233
+ libcxx=4.0.1=hcfea43d_1
+ libcxxabi=4.0.1=hcfea43d_1
+ libedit=3.1.20181209=hb402a30_0
+ libffi=3.2.1=h475c297_4
+ libgfortran=3.0.1=h93005f0_2
+ mkl=2019.4=233
+ mkl_fft=1.0.12=py37h5e564d8_0
+ mkl_random=1.0.2=py37h27c97d8_0
+ ncurses=6.1=h0a44026_1
+ numpy-base=1.16.4=py37h6575580_0
+ numpy=1.16.4=py37hacdab7b_0
+ openssl=1.1.1c=h1de35cc_1
+ pandas=0.24.2=py37h0a44026_0
+ python=3.7.3=h359304d_0
+ readline=7.0=h1de35cc_5
+ scipy=1.2.1=py37h1410ff5_0
+ sqlite=3.28.0=ha441bb4_0
+ statsmodels=0.10.0=py37h1d22016_0
+ tk=8.6.8=ha441bb4_0
+ xz=5.2.4=h1de35cc_4
+ zlib=1.2.11=h1de35cc_3
+ win-64:
+ icc_rt=2019.0.0=h0cc432a_1
+ intel-openmp=2019.4=245
+ mkl=2019.4=245
+ mkl_fft=1.0.12=py37h14836fe_0
+ mkl_random=1.0.2=py37h343c172_0
+ numpy-base=1.16.4=py37hc3f5095_0
+ numpy=1.16.4=py37h19fb1c0_0
+ openssl=1.1.1c=he774522_1
+ pandas=0.24.2=py37ha925a31_0
+ python=3.7.3=h8c8aaf0_1
+ scipy=1.2.1=py37h29ff71c_0
+ sqlite=3.28.0=he774522_0
+ statsmodels=0.10.0=py37h8c2d366_0
+ vc=14.1=h0510ff6_4
+ vs2015_runtime=14.15.26706=h3a45250_4
+ wincertstore=0.2=py37_0
Solving environment: ...working... done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.4
latest version: 4.7.5
Please update conda by running
$ conda update -n base conda
tk-8.6.8 | 3.1 MB | ########## | 100%
ca-certificates-2019 | 133 KB | ########## | 100%
mkl_random-1.0.2 | 405 KB | ########## | 100%
libgcc-ng-9.1.0 | 8.1 MB | ########## | 100%
intel-openmp-2019.4 | 876 KB | ########## | 100%
libstdcxx-ng-9.1.0 | 4.0 MB | ########## | 100%
numpy-base-1.16.4 | 4.4 MB | ########## | 100%
blas-1.0 | 6 KB | ########## | 100%
zlib-1.2.11 | 120 KB | ########## | 100%
setuptools-41.0.1 | 648 KB | ########## | 100%
pandas-0.24.2 | 11.1 MB | ########## | 100%
python-3.7.3 | 36.7 MB | ########## | 100%
mkl_fft-1.0.12 | 172 KB | ########## | 100%
xz-5.2.4 | 366 KB | ########## | 100%
patsy-0.5.1 | 375 KB | ########## | 100%
_libgcc_mutex-0.1 | 3 KB | ########## | 100%
mkl-2019.4 | 204.1 MB | ########## | 100%
pytz-2019.1 | 236 KB | ########## | 100%
openssl-1.1.1c | 3.8 MB | ########## | 100%
ncurses-6.1 | 958 KB | ########## | 100%
certifi-2019.6.16 | 154 KB | ########## | 100%
python-dateutil-2.8. | 281 KB | ########## | 100%
statsmodels-0.10.0 | 9.5 MB | ########## | 100%
numpy-1.16.4 | 49 KB | ########## | 100%
readline-7.0 | 392 KB | ########## | 100%
scipy-1.2.1 | 17.7 MB | ########## | 100%
libgfortran-ng-7.3.0 | 1.3 MB | ########## | 100%
six-1.12.0 | 22 KB | ########## | 100%
wheel-0.33.4 | 39 KB | ########## | 100%
sqlite-3.28.0 | 1.9 MB | ########## | 100%
libedit-3.1.20181209 | 188 KB | ########## | 100%
pip-19.1.1 | 1.8 MB | ########## | 100%
## Package Plan ##
environment location: /home/repl/mortgage_rates/envs/default
added / updated specs:
- _libgcc_mutex==0.1=main
- blas==1.0=mkl
- ca-certificates==2019.5.15=0
- certifi==2019.6.16=py37_0
- intel-openmp==2019.4=243
- libedit==3.1.20181209=hc058e9b_0
- libgcc-ng==9.1.0=hdf63c60_0
- libgfortran-ng==7.3.0=hdf63c60_0
- libstdcxx-ng==9.1.0=hdf63c60_0
- mkl==2019.4=243
- mkl_fft==1.0.12=py37ha843d7b_0
- mkl_random==1.0.2=py37hd81dba3_0
- ncurses==6.1=he6710b0_1
- numpy-base==1.16.4=py37hde5b4d6_0
- numpy==1.16.4=py37h7e9f1db_0
- openssl==1.1.1c=h7b6447c_1
- pandas==0.24.2=py37he6710b0_0
- patsy==0.5.1=py37_0
- pip==19.1.1=py37_0
- python-dateutil==2.8.0=py37_0
- python==3.7.3=h0371630_0
- pytz==2019.1=py_0
- readline==7.0=h7b6447c_5
- scipy==1.2.1=py37h7c811a0_0
- setuptools==41.0.1=py37_0
- six==1.12.0=py37_0
- sqlite==3.28.0=h7b6447c_0
- statsmodels==0.10.0=py37hdd07704_0
- tk==8.6.8=hbc83047_0
- wheel==0.33.4=py37_0
- xz==5.2.4=h14c3975_4
- zlib==1.2.11=h7b6447c_3
The following packages will be downloaded:
package | build
---------------------------|-----------------
tk-8.6.8 | hbc83047_0 3.1 MB
ca-certificates-2019.5.15 | 0 133 KB
mkl_random-1.0.2 | py37hd81dba3_0 405 KB
libgcc-ng-9.1.0 | hdf63c60_0 8.1 MB
intel-openmp-2019.4 | 243 876 KB
libstdcxx-ng-9.1.0 | hdf63c60_0 4.0 MB
numpy-base-1.16.4 | py37hde5b4d6_0 4.4 MB
blas-1.0 | mkl 6 KB
zlib-1.2.11 | h7b6447c_3 120 KB
setuptools-41.0.1 | py37_0 648 KB
pandas-0.24.2 | py37he6710b0_0 11.1 MB
python-3.7.3 | h0371630_0 36.7 MB
mkl_fft-1.0.12 | py37ha843d7b_0 172 KB
xz-5.2.4 | h14c3975_4 366 KB
patsy-0.5.1 | py37_0 375 KB
_libgcc_mutex-0.1 | main 3 KB
mkl-2019.4 | 243 204.1 MB
pytz-2019.1 | py_0 236 KB
openssl-1.1.1c | h7b6447c_1 3.8 MB
ncurses-6.1 | he6710b0_1 958 KB
certifi-2019.6.16 | py37_0 154 KB
python-dateutil-2.8.0 | py37_0 281 KB
statsmodels-0.10.0 | py37hdd07704_0 9.5 MB
numpy-1.16.4 | py37h7e9f1db_0 49 KB
readline-7.0 | h7b6447c_5 392 KB
scipy-1.2.1 | py37h7c811a0_0 17.7 MB
libgfortran-ng-7.3.0 | hdf63c60_0 1.3 MB
six-1.12.0 | py37_0 22 KB
wheel-0.33.4 | py37_0 39 KB
sqlite-3.28.0 | h7b6447c_0 1.9 MB
libedit-3.1.20181209 | hc058e9b_0 188 KB
pip-19.1.1 | py37_0 1.8 MB
------------------------------------------------------------
Total: 312.7 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex: 0.1-main
blas: 1.0-mkl
mkl_fft: 1.0.12-py37ha843d7b_0
mkl_random: 1.0.2-py37hd81dba3_0
numpy-base: 1.16.4-py37hde5b4d6_0
The following packages will be UPDATED:
ca-certificates: 2018.03.07-0 --> 2019.5.15-0
certifi: 2018.8.24-py36_1 --> 2019.6.16-py37_0
intel-openmp: 2018.0.0-hc7b2577_8 --> 2019.4-243
libedit: 3.1-heed3624_0 --> 3.1.20181209-hc058e9b_0
libgcc-ng: 7.2.0-h7cc24e2_2 --> 9.1.0-hdf63c60_0
libgfortran-ng: 7.2.0-h9f7466a_2 --> 7.3.0-hdf63c60_0
libstdcxx-ng: 7.2.0-h7a57d05_2 --> 9.1.0-hdf63c60_0
mkl: 2018.0.1-h19d6760_4 --> 2019.4-243
ncurses: 6.0-h9df7e31_2 --> 6.1-he6710b0_1
numpy: 1.14.1-py36h3dfced4_1 --> 1.16.4-py37h7e9f1db_0
openssl: 1.0.2p-h14c3975_0 --> 1.1.1c-h7b6447c_1
pandas: 0.22.0-py36hf484d3e_0 --> 0.24.2-py37he6710b0_0
patsy: 0.5.0-py36_0 --> 0.5.1-py37_0
pip: 9.0.1-py36h6c6f9ce_4 --> 19.1.1-py37_0
python: 3.6.4-hc3d631a_1 --> 3.7.3-h0371630_0
python-dateutil: 2.6.1-py36h88d3b88_1 --> 2.8.0-py37_0
pytz: 2018.3-py36_0 --> 2019.1-py_0
readline: 7.0-ha6073c6_4 --> 7.0-h7b6447c_5
scipy: 1.0.0-py36hbf646e7_0 --> 1.2.1-py37h7c811a0_0
setuptools: 38.5.1-py36_0 --> 41.0.1-py37_0
six: 1.11.0-py36h372c433_1 --> 1.12.0-py37_0
sqlite: 3.22.0-h1bed415_0 --> 3.28.0-h7b6447c_0
statsmodels: 0.8.0-py36h8533d0b_0 --> 0.10.0-py37hdd07704_0
tk: 8.6.7-hc745277_3 --> 8.6.8-hbc83047_0
wheel: 0.30.0-py36hfd4bba0_1 --> 0.33.4-py37_0
xz: 5.2.3-h55aa19d_2 --> 5.2.4-h14c3975_4
zlib: 1.2.11-ha838bed_2 --> 1.2.11-h7b6447c_3
Downloading and Extracting Packages
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
missing requirement to run this project: A downloaded file which is referenced by MORTGAGE_RATES.
Environment variable MORTGAGE_RATES is not set.
Added locked dependencies for env spec default to anaconda-project-lock.yml.
Project dependencies are locked.
In this exercise, you’ll create a compressed project archive and share it on Anaconda Cloud.
Note: You will need to create a free account on Anaconda Cloud to complete this exercise.
https://anaconda.org/#sign-up 申请账号即可。
$ anaconda-project archive ../mortgage_rates.zip
added mortgage_rates/.projectignore
added mortgage_rates/anaconda-project-lock.yml
added mortgage_rates/anaconda-project.yml
added mortgage_rates/forecast.py
added mortgage_rates/forecast.zip
Created project archive ../mortgage_rates.zip
The zip file will only include anaconda-project.yml and forecast.py since those two files can be used to recreate the package and data downloads. It is best practice not to create the zip archive within the project directory. Notice that envs and jpbAsR are not included in the archive.
并不是所有文件都在。
$ ls
anaconda-project-lock.yml envs forecast.zip
anaconda-project.yml forecast.py jpbAsR
$ ls ../
miniconda mortgage_rates mortgage_rates.zip
anaconda login --username datacamp-student --password datacamp1
登陆 anaconda 我估计需要配置 host
$ anaconda upload forecast.zip --package-type=project
Using Anaconda API: https://api.anaconda.org
Using "datacamp-student" as upload username
Uploading project: forecast
Using Anaconda API: https://api.anaconda.org
Upload successful.
Project forecast uploaded to https://anaconda.org/datacamp-student/project/forecast.
3.6 多个作者
目前 Python 包不如 R 包,支持多个作者,参考 add2pkg::add_me
。
multiple authors in the setup.py
when developing a python package
临时地,
Do they accept comma or semi-colon separated emails? Stack Overflow
使用;
加以区分。
I would recommend you limit these to a single author and mention all contributors in the documentation or description. Not every wish for a new “feature” should be satisfied, IMO. Besides, making metadata complex and version-dependent is the best way to make it useless. https://bugs.python.org/msg93286 Not every wish for a new “feature” should be satisfied, IMO. Besides, making metadata complex and version-dependent is the best way to make it useless. https://bugs.python.org/msg93284
但是官方认为,多个作者这种需要不应该放到目前的 metadata 来,可以写到 description,或者一个新文档。 因此不需要给多个作者。
参考主流的 numpy,也没多个作者。参考了 https://github.com/numpy/numpy 的确没有 DESC 这种。
DeFusco, Albert. 2019. “Conda for Building & Distributing Packages.” DataCamp. 2019. https://www.datacamp.com/courses/conda-for-building-distributing-packages.
Freddie Mac. 2019. “30-Year Fixed Rate Mortgage Average in the United States.” FRED, Federal Reserve Bank of St. Louis. 2019. https://fred.stlouisfed.org/series/MORTGAGE30US.