HoloViz project release procedure

TODO: turn into checklist

TODO: combine with the various similar procedures we've had around the place

TODO: flesh out the steps

TODO: completely draft

pyviz

Push dev tags:

$ git tag -a vX.Y.ZaN
$ git push --tags

to get packages on pyviz/label/dev until there's a satisfactory package. Or, if project is not on travis/you don't want to wait, you can build and test the package build locally, e.g.

Starting with a empty conda env and on a branch where all changes are committed:

# install pyctdev
$ conda install -c pyviz pyctdev && doit ecosystem=conda ecosystem_setup

# build pyct-core and run the 'build_examples' tests (defined in tox.ini) with python 3.6 
$ doit ecosystem=conda package_build --recipe=core --test-python=py36 --test-group=build_examples -c pyviz
 
# build pyct and run the 'cmd_examples' tests (defined in tox.ini) with python 2.7 
$ doit ecosystem=conda package_build --test-python=py27 --test-group=cmd_examples -c pyviz

Packages are tested automatically, but only by unit tests and running a limited set of the notebooks. Additionally, the uploaded packages are not yet tested on other platforms (even if the regular tests are run on other platforms; https://github.com/pyviz/pyct/issues/7).

Therefore, to determine if a package is satisfactory, some manual testing is required. Get the package, copy the examples, and download the data:

$ cd /scratch

# conda
$ conda create -n releasetest python=2.7
$ conda activate releasetest
$ conda install -c pyviz/label/dev datashader
$ datashader examples

# or pip
$ python -m venv ./releasetest
$ source ./releasetest/bin/activate
$ pip install --pre --upgrade --index-url=https://test.pypi.org/simple --extra-index-url=https://pypi.org/simple datashader
$ datashader examples


$ datashader --version # matches what you expect?
$ cd datashader-examples

Things to try:

• conda/pip install jupyter notebook and try out the notebooks

• try out the apps

• Run all the notebooks (not just the limited set run during automated testing). Install pytest flake8 nbsmoke to get the test dependencies, then pytest --nbsmoke-run -k ".ipynb" --ignore-nbsmoke-skip-run

• Open and inspect the packages from https://test.pypi.org/project/datashader/ and https://anaconda.org/pyviz/datashader

What can go wrong between dev tag and release tag?

The dev packages are built and tested using pyviz/label/dev. That means they get other pyviz/label/dev packages. The release packages are built and tested using pyviz. That means they get other pyviz packages (not dev). So you can imagine a dev package being fine, but a release package failing, because of dependencies.

Again, you can try locally if you prefer, e.g.

$ doit package_build --recipe=core -c pyviz --test-python=py27 --test-group=unit && doit package_build --recipe=recommended -c pyviz --test-python=py36 --test-group=examples

conda-forge

Once a release tag has been pushed, mature projects probably won't have to do anything for conda-forge. A PR will be opened automatically by conda-forge, and the package will be built, tested (but see https://github.com/pyviz/pyct/issues/25). Someone merge the PR to make it available.

Fast changing projects or those with tricky/many dependencies will likely need to update the c-f recipe. To do this, take the recipe from the pyviz package (not the templated version in the repository, but the rendered recipe in the package), and use it as reference for updating dependencies/pins etc.

defaults

Once on c-f, ? file issue for defaults to update their fork of c-f ?

---

datashader (needs merge)

NOTE: remove everything from examples/data prior to building

• Ensure git repository is clear of unmerged/untracked files.

• Ensure all tests pass.

• Update meta.yaml and examples/environment.yml with best current recommended dependencies (often these float during development, but need to be pinned for releases).

• Update version number in conda.recipe/meta.yaml, datashader/__init__.py, and setup.py. Commit.

• Tag commit and push to github

git tag -a x.x.x -m 'Version x.x.x'
git push origin master --tags

• Build conda packages

The exact procedure is platform/setup specific, so I'll define a few variables here, to fill in with your specifics:

# Location of your conda install. For me it's `~/anaconda/`
CONDA_DIR=~/anaconda/

# Platform code. For me it's `osx-64`
PLATFORM=osx-64

# Version number of datashader being released (e.g. 0.2.0)
VERSION=0.2.0

This assumes conda, conda-build, and anaconda-client are installed (if not, install conda, then use conda to install the others). From inside the toplevel directory:

conda config --append channels ioam
conda build conda.recipe/ --python 2.7
conda build conda.recipe/ --python 3.5
conda build conda.recipe/ --python 3.6

Next, cd into the folder where the builds end up.

cd $CONDA_DIR/conda-bld/$PLATFORM

Use conda convert to convert over the missing platforms (skipping the one for the platform you're currently on):

conda convert --platform osx-64 datashader-$VERSION*.tar.bz2 -o ../
conda convert --platform linux-64 datashader-$VERSION*.tar.bz2 -o ../
conda convert --platform win-64 datashader-$VERSION*.tar.bz2 -o ../

Use anaconda upload to upload the build to the bokeh channel. This requires you to be setup on anaconda.org, and have the proper credentials to push to the bokeh channel.

anaconda login
anaconda upload $CONDA_DIR/conda-bld/*/datashader-$VERSION*.tar.bz2 -u bokeh

If you are uploading a dev release, then add the dev label.

anaconda upload $CONDA_DIR/conda-bld/*/datashader-$VERSION*.tar.bz2 -u bokeh -l dev

• Write the release notes:

1. Run git log to get a listing of all the changes
2. Remove any covered in the previous release
3. Summarize the rest to focus on user-visible changes and major new features
4. Paste the notes into github, under n releases, then Tags, then Edit release notes.