A more comprehensive release procedure.

The old release procedure did not mention explicitly some
necessary steps and generally required a lot of thinking.

The new version is an attempt to create a procedure which can
be followed from top to bottom without too much thinking.

Author: sbarzowski

Diff for: release_checklist.md

@@ -1,47 +1,114 @@
-# Checklist for building Jsonnet releases
-
-## Are jsonnet & go-jsonnet in sync?
-1) Check out master from both
-1) Check go-jsonnet/cpp-jsonnet submodule is HEAD of jsonnet tree, if not:
-1) Inside cpp-jsonnet:
-1) `git checkout master`
-1) `git pull`
-1) `cd ..`
-1) `git checkout -b release`
-1) `git commit -a -m 'update cpp-jsonnet'`
-1) `./tests.sh`  # Have to commit before running this, or it resets the submodule
-
-## Is HEAD ready?
-1) Checkout master
-1) `make test`
-1) `bazel test ...:all`
-1) `mkdir build ; cd build ; cmake .. ; make ; make test`
-1) modify `include/libjsonnet.h` to bump the version number but add `-pre1`
-1)
+# Jsonnet Release Checklist
+
+## Preparation
+
+Before doing the release, make sure that the project is in good state:
+
+* Triage issues – make sure that all issues are labelled.
+* Check for release-blocking bugs. All bugs which results in a wrong result of the evaluation are release-blocking.
+* Go through the open PRs – consider merging any outstanding ones. Do not merge big changes right before the release.
+* Sync google/go-jsonnet and google/jsonnet
+    1) Check out master from both
+    1) Check go-jsonnet/cpp-jsonnet submodule is HEAD of jsonnet tree, if not:
+    1) Inside cpp-jsonnet:
+    1) `git checkout master`
+    1) `git pull`
+    1) `cd ..`
+    1) `git checkout -b release`
+    1) Update the standard library: `go run cmd/dumpstdlibast/dumpstdlibast.go cpp-jsonnet/stdlib/std.jsonnet > astgen/stdast.go`
+    1) `./tests.sh`
+    1) Fix any failing tests.
+    1) Send a PR with the updated version, so that all the CI tests are run
+* Check that CI is green.
+* Make sure that the project can be built and the tests pass using all build systems.
+    * C++:
+        1) `make test`
+        1) `bazel test ...:all`
+        1) `mkdir build ; cd build ; cmake .. ; make ; make test`
+    * Go:
+        1) `./tests.sh`
+        2) `bazel test //:go_default_test`
+* Make sure the Python bindings work (build and run tests locally).
+* Go through the commits since the last release and prepare the release notes.
+    * The release notes should have separate sections for the language changes and
+    changes specific to each implementation.
+* Make sure that the stdlib documentation is complete. Check the release notes for any
+additions or changes to stdlib and make sure they are reflected in the documentation.
+* Make sure that you can build the website locally.
+    * `make doc/js/libjsonnet.js`
+    * `tools/scripts/serve_docs.sh`
+    * Check that it works in two different browsers. Make sure that live evaluation
+    in the tutorial works.
+
+All the above points apply to both google/jsonnet and google/go-jsonnet.
+
+## Make a final decision to release
+
+A this point you should be confident that the project is ready for the release.
+Some of subsequent steps leave project are hard to reverse and/or leave the project in
+an inconsistent state, so proceed only if you have time to complete the release process
+in one sitting and fix any unexpected problems.
+
+## Release the C++ version
+
+* Modify `include/libjsonnet.h` to bump the version number but add `-pre1` suffix.
+* Update `test_cmd` golden file version numbers:
+```
+./tools/scripts/replace_test_cmd_version.sh <NEW_VERSION>-pre1
 ```
-find test_cmd -name '*.cpp' -o -name '*.golang' -o -name '*.stdout' -o -name '*.stderr' -o -name 'stdout' -o -name 'stderr' | \
-  xargs sed -i 's/ v0[.][0-9.]*/ NEW_VERSION_GOES_HERE/g'
+* Upload and test the pre-release Python package.
+    1) `python setup.py build sdist`
+    1) `twine upload dist/whatever.tar.gz`  (Needs credentials on pypi)
+    1) ON ANOTHER MACHINE AND CHECK THAT THE VERSION IS CORRECT AND IT ACTUALLY IS BUILDING THINGS IN THE LOG: `sudo pip install jsonnet --pre --upgrade`
+    1) `python`
+    1) `import _jsonnet`
+    1) `_jsonnet.evaluate_snippet('foo', '1+1')`
+* Remove -pre from version in `include/libjsonnet.h`.
+* Update version in `cpp/CMakeLists.txt`.
+* Update `test_cmd` golden file version numbers:
 ```
-1) Check if any changes to the documentation are necessary by checking commits since previous release (especially stdlib additions).
-1) In stdlib documentation replace any "Available in upcoming release." with "Available since NEW_VERSION_GOES_HERE".
-1) `python setup.py build sdist`
-1) `twine upload dist/whatever.tar.gz`  (Needs credentials on pypi)
-1) ON ANOTHER MACHINE AND CHECK THAT THE VERSION IS CORRECT AND IT ACTUALLY IS BUILDING THINGS IN THE LOG: `sudo pip install jsonnet --pre --upgrade`
-1) `python`
-1) `import _jsonnet`
-1) `_jsonnet.evaluate_snippet('foo', '1+1')`
-1) Checkout go-jsonnet master
-1) update cpp-jsonnet commit
-1) `./tests`
-
-## Do release
-1) Remove -pre from version in `include/libjsonnet.h`
-1) Update version in `CMakeLists.txt`
-1) Update `test_cmd` golden file version numbers
-1) run test again!
-1) commit and push
-1) Make release in github, write release notes by checking commits since previous release
-1) `python setup.py build sdist`
-1) `twine upload dist/whatever.tar.gz` (the version without the -pre)
-1) Post to Jsonnet Google Group
-1) Update the live version of the website
+./tools/scripts/replace_test_cmd_version.sh <NEW_VERSION>
+```
+* In stdlib documentation replace any "Available in upcoming release." with "Available since NEW_VERSION_GOES_HERE".
+* Send a PR and wait for CI to be green before merging.
+* Create a release on Github. Put the previously prepared release notes there.
+* Upload the Python package.
+    1) `python setup.py build sdist`
+    1) `twine upload dist/whatever.tar.gz` (the version without the -pre)
+
+## Release the Go version
+
+* Sync with the (freshly-released) C++ version.
+* At this point the C++ tests will not pass due to a different version in test_cmd tests.
+* Modify `vm.go` to bump the version number, but add `-pre1` suffix.
+* Upload and test the pre-release Python package.
+    1) `python setup.py build sdist`
+    1) `twine upload dist/whatever.tar.gz`  (Needs credentials on pypi)
+    1) ON ANOTHER MACHINE AND CHECK THAT THE VERSION IS CORRECT AND IT ACTUALLY IS BUILDING THINGS IN THE LOG: `sudo pip install gojsonnet --pre --upgrade`
+    1) `python`
+    1) `import _gojsonnet`
+    1) `_gojsonnet.evaluate_snippet('foo', '1+1')`
+* Remove the `-pre1` from version in `vm.go`. Now the tests should pass again.
+* Send a PR and wait for CI to be green before merging.
+* Create a release on Github. Link to the C++ release in the description.
+* Upload the Python package.
+    1) `python setup.py build sdist`
+    1) `twine upload dist/whatever.tar.gz` (the version without the -pre)
+
+## Update the website
+
+In google/jsonnet:
+* `make doc/js/libjsonnet.js`
+* `tools/scripts/serve_docs.sh`
+* Check that the local version works in two different browsers. Make sure that live evaluation in the tutorial works.
+* `tools/scripts/push_docs.sh`
+* Check that the public works in two different browsers. Make sure that you are getting the new version (and not an old cached version). Make sure that live evaluation in the tutorial works.
+
+## Make the announcement
+
+* Send an email to the mailing list.
+* Announce the new release on Slack.
+
+## After the release
+
+It's a good time to merge big PRs.

Diff for: tools/scripts/replace_test_cmd_version.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+set -e
+
+if [ "$#" != 1 ]; then
+    echo "Usage: $0 <NEW_VERSION>"
+fi
+
+NEW_VERSION="$1"
+
+set -x
+
+find test_cmd -name '*.cpp' -o -name '*.golang' -o -name '*.stdout' -o -name '*.stderr' | \
+	xargs sed -i 's/ v0[.][0-9.]*\(-pre[0-9]*\)\{0,1\}/ '"$NEW_VERSION"'/g'