Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
369 lines (264 loc) · 11.8 KB

CONTRIBUTING.md

File metadata and controls

369 lines (264 loc) · 11.8 KB

HOW TO CONTRIBUTE TO TQDM

TL;DR: Skip to [QUICK DEV SUMMARY]

This file describes how to

  • contribute changes to the project, and
  • upload released to the PyPI repository.

Most of the management commands have been directly placed inside the Makefile:

make [<alias>]  # on UNIX-like environments
python -m pymake [<alias>]  # if make is unavailable

The latter depends on py-make>=0.1.0.

Use the alias help (or leave blank) to list all available aliases.

HOW TO COMMIT CONTRIBUTIONS

Contributions to the project are made using the "Fork & Pull" model. The typical steps would be:

  1. create an account on github
  2. fork tqdm
  3. make a local clone: git clone https://github.com/your_account/tqdm.git
  4. make changes on the local copy
  5. test (see below) and commit changes git commit -a -m "my message"
  6. push to your GitHub account: git push origin
  7. create a Pull Request (PR) from your GitHub fork (go to your fork's webpage and click on "Pull Request." You can then add a message to describe your proposal.)

WHAT CODE LAYOUT SHOULD I FOLLOW?

Don't worry too much - maintainers can help reorganise contributions. However it would be helpful to bear in mind:

  • The standard core of tqdm, i.e. tqdm.std.tqdm
    • must have no dependencies apart from pure python built-in standard libraries
    • must have negligible impact on performance
    • should have 100% coverage by unit tests
    • should be appropriately commented
    • should have well-formatted docstrings for functions
      • under 76 chars (incl. initial spaces) to avoid linebreaks in terminal pagers
      • use two spaces between variable name and colon, specify a type, and most likely state that it's optional: VAR<space><space>:<space>TYPE[, optional]
      • use [default: ...] for default values of keyword arguments
    • will not break backward compatibility unless there is a very good reason
      • e.g. breaking py26 compatibility purely in favour of minor readability changes (such as converting dict(a=1) to {'a': 1}) is not a good enough reason
    • API changes should be discussed carefully
    • remember, with millions of downloads per month, tqdm must be extremely fast and reliable
  • Any other kind of change may be included in a (possibly new) submodule
    • submodules are likely single python files under the main tqdm/ directory
    • submodules extending tqdm.std.tqdm or any other module (e.g. tqdm.notebook.tqdm, tqdm.gui.tqdm)
    • CLI wrapper tqdm.cli
      • if a newly added tqdm.std.tqdm option is not supported by the CLI, append to tqdm.cli.UNSUPPORTED_OPTS
    • can implement anything from experimental new features to support for third-party libraries such as pandas, numpy, etc.
    • submodule maturity
      • alpha: experimental; missing unit tests, comments, and/or feedback; raises tqdm.TqdmExperimentalWarning
      • beta: well-used; commented, perhaps still missing tests
      • stable: >10 users; commented, 80% coverage
  • .meta/
    • A "hidden" folder containing helper utilities not strictly part of the tqdm distribution itself

TESTING

Once again, don't worry too much - tests are automated online, and maintainers can also help.

To test functionality (such as before submitting a Pull Request), there are a number of unit tests.

Standard unit tests

The standard way to run the tests:

  • install tox
  • cd to the root of the tqdm directory (in the same folder as this file)
  • run the following command:
[python -m py]make test
# or:
tox --skip-missing-interpreters

This will build the module and run the tests in a virtual environment. Errors and coverage rates will be output to the console/log. (Ignore missing interpreters errors - these are due to the local machine missing certain versions of Python.)

Note: to install all versions of the Python interpreter that are specified in tox.ini, you can use MiniConda to install a minimal setup. You must also ensure that each distribution has an alias to call the Python interpreter (e.g. python311 for Python 3.11's interpreter).

Alternative unit tests with pytest

Alternatively, use pytest to run the tests just for the current Python version:

  • install test requirements: [python -m py]make install_test
  • run the following command:
[python -m py]make alltests

MANAGE A NEW RELEASE

This section is intended for the project's maintainers and describes how to build and upload a new release. Once again, [python -m py]make [<alias>] will help. Also consider pip installing development utilities: [python -m py]make install_build at a minimum, or a more thorough conda env create.

Pre-commit Hook

It's probably a good idea to use the pre-commit (pip install pre-commit) helper.

Run pre-commit install for convenient local sanity-checking.

Semantic Versioning

The tqdm repository managers should:

Checking pyproject.toml

To check that the pyproject.toml file is compliant with PyPI requirements (e.g. version number; reStructuredText in README.rst) use:

[python -m py]make testsetup

To upload just metadata (including overwriting mistakenly uploaded metadata) to PyPI, use:

[python -m py]make pypimeta

Merging Pull Requests

This section describes how to cleanly merge PRs.

1 Rebase

From your project repository, merge and test (replace pr-branch-name as appropriate):

git fetch origin
git checkout -b pr-branch-name origin/pr-branch-name
git rebase master

If there are conflicts:

git mergetool
git rebase --continue

2 Push

Update branch with the rebased history:

git push origin pr-branch-name --force

Non maintainers can stop here.

Note: NEVER just git push --force (this will push all local branches, overwriting remotes).

3 Merge

git checkout master
git merge --no-ff pr-branch-name

4 Test

[python -m py]make alltests

5 Push to master

git push origin master

Building a Release and Uploading to PyPI

Formally publishing requires additional steps: testing and tagging.

Test

Ensure that all online CI tests have passed.

Tag

  • ensure the version has been tagged. The tag format is v{major}.{minor}.{patch}, for example: v4.4.1. The current commit's tag is used in the version checking process. If the current commit is not tagged appropriately, the version will display as v{major}.{minor}.{patch}.dev{N}+g{commit_hash}.

Upload

GitHub Actions (GHA) CI should automatically do this after pushing tags. Manual instructions are given below in case of failure.

Build tqdm into a distributable python package:

[python -m py]make build

This will generate several builds in the dist/ folder. On non-windows machines the windows exe installer may fail to build. This is normal.

Finally, upload everything to PyPI. This can be done easily using the twine module:

[python -m py]make pypi

Also, the new release can (should) be added to GitHub by creating a new release from the web interface; uploading packages from the dist/ folder created by [python -m py]make build. The wiki can be automatically updated with GitHub release notes by running make within the wiki repository.

Docker images may be uploaded to https://hub.docker.com/r/tqdm/tqdm. Assuming docker is installed:

make -B docker
docker login
docker push tqdm/tqdm:latest
docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)

Snaps may be uploaded to https://snapcraft.io/tqdm. Assuming snapcraft is installed (snap install snapcraft --classic --beta):

make snap
snapcraft login
snapcraft push tqdm*.snap --release stable

Notes

  • you can also test on the PyPI test servers test.pypi.org before the real deployment
  • in case of a mistake, you can delete an uploaded release on PyPI, but you cannot re-upload another with the same version number
  • in case of a mistake in the metadata on PyPI (e.g. bad README), updating just the metadata is possible: [python -m py]make pypimeta

Updating Websites

The most important file is .readme.rst, which should always be kept up-to-date and in sync with the in-line source documentation. This will affect all of the following:

  • README.rst (generated by mkdocs.py during make build)
  • The main repository site which automatically serves the latest README.rst as well as links to all of GitHub's features. This is the preferred online referral link for tqdm.
  • The PyPI mirror which automatically serves the latest release built from README.rst as well as links to past releases.
  • Many external web crawlers.

Additionally (less maintained), there exists:

Helper Bots

There are some helpers in .github/workflows to assist with maintenance.

  • Comment Bot
    • allows maintainers to write /tag vM.m.p commit_hash in an issue/PR to create a tag
  • Post Release
  • Benchmark

QUICK DEV SUMMARY

For experienced devs, once happy with local master, follow the steps below. Much is automated so really it's steps 1-5, then 11(a).

  1. test ([python -m py]make alltests or rely on pre-commit)
  2. git commit [--amend] # -m "bump version"
  3. git push
  4. wait for tests to pass a) in case of failure, fix and go back to (1)
  5. git tag vM.m.p && git push --tags or comment /tag vM.m.p commit_hash
  6. [AUTO:GHA] [python -m py]make distclean
  7. [AUTO:GHA] [python -m py]make build
  8. [AUTO:GHA] upload to PyPI. either: a) [python -m py]make pypi, or b) twine upload -s -i $(git config user.signingkey) dist/tqdm-*
  9. [AUTO:GHA] upload to docker hub: a) make -B docker b) docker push tqdm/tqdm:latest c) docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)
  10. [AUTO:GHA] upload to snapcraft: a) make snap, and b) snapcraft push tqdm*.snap --release stable
  11. Wait for GHA to draft a new release on https://github.com/tqdm/tqdm/releases a) replace the commit history with helpful release notes, and click publish b) [AUTO:GHA] attach dist/tqdm-* binaries (usually only *.whl*)
  12. [SUB][AUTO:GHA-rel] run make in the wiki submodule to update release notes
  13. [SUB][AUTO:GHA-rel] run make deploy in the docs submodule to update website
  14. [SUB][AUTO:GHA-rel] accept the automated PR in the feedstock submodule to update conda
  15. [AUTO:GHA-rel] update the gh-pages project benchmarks a) [python -m py]make testasvfull b) asv gh-pages

Key:

  • [AUTO:GHA]: GitHub Actions CI should automatically do this after git push --tags (5)
  • [AUTO:GHA-rel]: GitHub Actions CI should automatically do this after release (11a)
  • [SUB]: Requires one-time make submodules to clone docs, wiki, and feedstock