development.md

Development

This document is a summary of how to do various tasks one runs into as a developer of Cirq. Note that all commands assume a Debian environment, and all commands (except the initial repository cloning command) assume your current working directory is the cirq repo root.

Cloning the repository

The simplest way to get a local copy of cirq that you can edit is by cloning Cirq's github repository:

git clone git@github.com:quantumlib/cirq.git
cd cirq

To do your development in a Docker image, you can build one with Cirq/dev_tools/Dockerfile or pull an existing image:

    docker pull quantumlib/cirq:dev
    docker run -it quantumlib/cirq:dev python -c "import cirq; print(cirq.google.Foxtail)"

If you want to contribute changes to Cirq, you will instead want to fork the repository and submit pull requests from your fork.

Forking the repository

1. Fork the Cirq repo (Fork button in upper right corner of repo page). Forking creates a new github repo at the location https://github.com/USERNAME/cirq where USERNAME is your github id.
2. Clone the fork you created to your local machine at the directory where you would like to store your local copy of the code, and cd into the newly created directory.

   git clone git@github.com:USERNAME/cirq.git
   cd cirq

   (Alternatively, you can clone the repository using the URL provided on your repo page under the green "Clone or Download" button)
3. Add a remote called upstream to git. This remote will represent the main git repo for cirq (as opposed to the clone, which you just created, which will be the origin remote). This remote can be used to merge changes from Cirq's main repository into your local development copy.

   git remote add upstream https://github.com/quantumlib/cirq.git

   To verify the remote, run git remote -v. You should see both the origin and upstream remotes.
4. Sync up your local git with the upstream remote:

   git fetch upstream

   You can check the branches that are on the upstream remote by running git remote -va or git branch -r. Most importantly you should see upstream/master listed.
5. Merge the upstream master into your local master so that it is up to date.

   git checkout master
   git merge upstream/master

   At this point your local git master should be synced with the master from the main cirq repo.

Setting up an environment

0. First clone the repository, if you have not already done so. See the previous section for instructions.

1. Install system dependencies.

   Make sure you have python 3.5 or greater. You can install most other dependencies via apt-get:

   cat apt-system-requirements.txt dev_tools/conf/apt-list-dev-tools.txt | xargs sudo apt-get install --yes

   There are some extra steps if protocol buffers are changed; see the next section.

2. Prepare a virtual environment including the dev tools (such as mypy).

   One of the system dependencies we installed was virtualenvwrapper, which makes it easy to create virtual environments. If you did not have virtualenvwrapper previously, you may need to re-open your terminal or run source ~/.bashrc before these commands will work:

   mkvirtualenv cirq-py3 --python=/usr/bin/python3
   python -m pip install --upgrade pip
   python -m pip install -e .[dev_env]

   (When you later open another terminal, you can activate the virtualenv with workon cirq-py3.)

3. Check that the tests pass.

   pytest .

4. (OPTIONAL) include your development copy of cirq in your python path.

   PYTHONPATH="$(pwd)":"${PYTHONPATH}"

   or add it to the python path, but only in the virtualenv.

   add2virtualenv ./

Protocol buffers

Protocol buffers are used in Cirq for converting circuits, gates, and other objects into a standard form that can be written and read by other programs. Cirq's protobufs live at cirq/api/google and may need to be changed or extended from time to time.

If any protos are updated, their dependents can be rebuilt by calling the script dev_tools/build-protos.sh. This script uses grpcio-tools and protobuf version 3.8.0 to generate the python proto api.

Additionally, for workflows that use bazel (relevant for C/C++ code depending on Cirq), we have made available bazel rulesets for generating both python and C/C++ proto apis. These rules live in the BUILD files here and here. Downstream projects should load Cirq as an external dependency, allowing rules from those BUILD files to be used directly.

Continuous integration and local testing

There are a few options for running continuous integration checks, varying from easy and fast to slow and reliable.

The simplest way to run checks is to invoke pytest, pylint, or mypy for yourself as follows:

pytest
pylint --rcfile=dev_tools/conf/.pylintrc cirq
mypy --config-file=dev_tools/conf/mypy.ini .

This can be a bit tedious, because you have to specify the configuration files each time. A more convenient way to run checks is to via the scripts in the check/ directory, which specify configuration arguments for you and cover more use cases:

# Run all tests in the repository.
./check/pytest [files-and-flags-for-pytest]

# Check all relevant files in the repository for lint.
./check/pylint [files-and-flags-for-pylint]

# Typecheck all python files in the repository.
./check/mypy [files-and-flags-for-mypy]

# Compute incremental coverage vs master (or a custom revision of your choice).
./check/pytest-and-incremental-coverage [BASE_REVISION]

# Only run tests associated with files that have changed when diffed vs master (or a custom revision of your choice).
./check/pytest-changed-files [BASE_REVISION]

# Run the documentation tests.
./check/doctest

# Check the format of the filess.  Use --apply to apply the suggested format changes.
./check/format-incremental [--apply]

# Run all of the above tests. Which pytest is run is set by the --only-changed-files.
./check/all [BASE_REVISION] [--only-changed-files] [--apply-format-changes]

The above scripts are convenient and reasonably fast, but they often won't exactly match the results computed by the continuous integration builds run on T ravis. For example, you may be running an older version of pylint or numpy. If you need to test against the actual continuous integration check, open up a pull request. For this pull request you may want to mark it as [Testing] so that it is not reviewed.

Writing docstrings and generating documentation

Cirq uses Google style doc strings with a markdown flavor and support for latex. Here is an example docstring:

def some_method(a: int, b: str) -> float:
    r"""One line summary of method.

    Additional information about the method, perhaps with some sort of latex
    equation to make it clearer:

        $$
        M = \begin{bmatrix}
                0 & 1 \\
                1 & 0
            \end{bmatrix}
        $$

    Notice that this docstring is an r-string, since the latex has backslashes.
    We can also include example code:

        print(cirq.google.Foxtail)

    You can also do inline latex like $y = x^2$ and inline code like
    `cirq.unitary(cirq.X)`.

    And of course there's the standard sections.

    Args:
        a: The first argument.
        b: Another argument.

    Returns:
        An important value.

    Raises:
        ValueError: The value of `a` wasn't quite right.
    """

Documentation is generated automatically by readthedocs when pushing to master, but you can also generated a local copy by running:

dev_tools/build-docs.sh

The HTML output will go into the docs/_build directory.

Producing a pypi package

1. Do a dry run with test pypi.

   If you're making a release, you should have access to a test pypi account capable of uploading packages to cirq. Put its credentials into the environment variables TEST_TWINE_USERNAME and TEST_TWINE_PASSWORD then run

   ./dev_tools/packaging/publish-dev-package.sh EXPECTED_VERSION --test

   You must specify the EXPECTED_VERSION argument to match the version in cirq/_version.py, and it must contain the string dev. This is to prevent accidentally uploading the wrong version.

   The script will append the current date and time to the expected version number before uploading to test pypi. It will print out the full version that it uploaded. Take not of this value.

   Once the package has uploaded, verify that it works

   ./dev_tools/packaging/verify-published-package.sh FULL_VERSION_REPORTED_BY_PUBLISH_SCRIPT --test

   The script will create fresh virtual environments, install cirq and its dependencies, check that code importing cirq executes, and run the tests over the installed code. If everything goes smoothly, the script will finish by printing VERIFIED.

2. Do a dry run with prod pypi

   This step is essentially identical to the test dry run, but with production pypi. You should have access to a production pypi account capable of uploading packages to cirq. Put its credentials into the environment variables PROD_TWINE_USERNAME and PROD_TWINE_PASSWORD then run

   ./dev_tools/packaging/publish-dev-package.sh EXPECTED_VERSION --prod

   Once the package has uploaded, verify that it works

   ./dev_tools/packaging/verify-published-package.sh FULL_VERSION_REPORTED_BY_PUBLISH_SCRIPT --prod

   If everything goes smoothly, the script will finish by printing VERIFIED.

3. Set the version number in cirq/_version.py.

   Development versions end with .dev or .dev#. For example, 0.0.4.dev500 is a development version of the release version 0.0.4. For a release, create a pull request turning #.#.#.dev* into #.#.# and a follow up pull request turning #.#.# into (#+1).#.#.dev.

4. Run dev_tools/packaging/produce-package.sh to produce pypi artifacts.

   dev_tools/packaging/produce-package.sh dist

   The output files will be placed in the directory dist/.

5. Create a github release.

   Describe major changes (especially breaking changes) in the summary. Make sure you point the tag being created at the one and only revision with the non-dev version number. Attach the package files you produced to the release.

6. Upload to pypi.

   You can use a tool such as twine for this. For example:

   twine upload -u "${PROD_TWINE_USERNAME}" -p "${PROD_TWINE_PASSWORD}" dist/*

   You should then run the verification script to check that the uploaded package works:

   ./dev_tools/packaging/verify-published-package.sh VERSION_YOU_UPLOADED --prod

   And try it out for yourself:

   python -m pip install cirq
   python -c "import cirq; print(cirq.google.Foxtail)"
   python -c "import cirq; print(cirq.__version__)"