Skip to content

Ahzyuan/Python-package-template

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

18 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ“ฆ A Project Template for Self-developed Python Package

Package Version License Pypi Template

setuptools Ruff Isort

โ€ข Planning to develop your first Python third-party package?
โ€ข Troubled by setuptools's numerous, complex configurations?
โ€ข Unsure about what the structure of a project should be?
๐“๐ก๐ž๐ง ๐ฒ๐จ๐ฎ'๐ฏ๐ž ๐œ๐จ๐ฆ๐ž ๐ญ๐จ ๐ญ๐ก๐ž ๐ซ๐ข๐ ๐ก๐ญ ๐ฉ๐ฅ๐š๐œ๐ž!

This repo provides an ๐จ๐ฎ๐ญ-๐จ๐Ÿ-๐ญ๐ก๐ž-๐›๐จ๐ฑ ๐ฉ๐ซ๐จ๐ฃ๐ž๐œ๐ญ ๐ฌ๐ญ๐ซ๐ฎ๐œ๐ญ๐ฎ๐ซ๐ž ๐ญ๐ž๐ฆ๐ฉ๐ฅ๐š๐ญ๐ž that accelerates your third-party Python package development.

๐ŸŽฏ Features

๐๐ซ๐š๐œ๐ญ๐ข๐œ๐š๐ฅ, ๐š๐ง๐ ๐ซ๐ž๐š๐๐ฒ ๐ญ๐จ ๐ ๐จ ๐ฌ๐ญ๐ซ๐š๐ข๐ ๐ก๐ญ ๐จ๐ฎ๐ญ ๐จ๐Ÿ ๐ญ๐ก๐ž ๐›๐จ๐ฑ

๐Ÿ’ก Tips
โ€ข We use setup.cfg to manage all metadata, and just keep a minimal setup.py to ensure editable installation supported.

We provide:

  1. A fully configured package-setup file, i.e., setup.cfg or pyproject.toml.

    • It covers most common config items, allows dynamic access to version, README, and project dependencies when building.
    • It is well commented, so you don't need to look up documents to understand each item's meaning.
  2. A complete and concise usage guidance, i.e. the ๐Ÿ”จ Usage section below.

  3. CI/CD support: Once a push with a tag is made and the tag matches a template of the form v*.*.*, the CI/CD pipeline will be triggered to build the package, upload it to PyPI and TestPyPI and create a release in your github project according to tag name and CHANGELOG.md. See the CI/CD via Github Action ๐Ÿค– section below.

๐„๐Ÿ๐Ÿ๐ข๐œ๐ข๐ž๐ง๐ญ ๐š๐ง๐ ๐ฉ๐ซ๐จ๐Ÿ๐ž๐ฌ๐ฌ๐ข๐จ๐ง๐š๐ฅ

We provide a useful, complete project structure, which
โ€ข not only complies with software engineering specifications,
โ€ข but also includes all file templates required for a project and continuous deployment(CD) workflows(see the CI/CD via Github Action ๐Ÿค– section below).

Here is the detailed structure of the project:

Python-package-template/
โ”œโ”€โ”€ .github/                      # Store Github Action workflow files and templates of Issue, PR 
โ”‚   โ”œโ”€โ”€ CONTRIBUTING.md           # Instructions for contributing to project
โ”‚   โ”œโ”€โ”€ ISSUE_TEMPLATE            # Store Issue template files
โ”‚   โ”‚   โ”œโ”€โ”€ bug_report.yml        # Bug report template
โ”‚   โ”‚   โ”œโ”€โ”€ feature_request.yml   # Feature request template
โ”‚   โ”‚   โ””โ”€โ”€ config.yml            # Template choosing configuration
โ”‚   โ”œโ”€โ”€ PULL_REQUEST_TEMPLATE.md  # Template for PR description
โ”‚   โ””โ”€โ”€ workflows                 # Store Github Action workflow files    
โ”‚       โ””โ”€โ”€ publish_release.yml   # Workflow for publishing and releaseing Python package
|
โ”œโ”€โ”€ tests/           # Store testing code
โ”‚   โ””โ”€โ”€ README.md    # Instructions of how to test your code
|
โ”œโ”€โ”€ docs/            # Store document related files
โ”‚   โ””โ”€โ”€ README.md    # Instructions of how to build document for your project
|
โ”œโ”€โ”€ examples/        # Store project demo code
โ”‚   โ””โ”€โ”€ demo.ipynb   # Demonstration of your project
|
โ”œโ”€โ”€ package-name/    # Store project code
โ”‚   โ”œโ”€โ”€ core.py      # Core code
โ”‚   โ””โ”€โ”€ __init__.py  # Package initialization file, defining copyright, version,and other information
|
โ”œโ”€โ”€ .gitignore       # File patterns which will be ignored by Git
โ”œโ”€โ”€ LICENSE          # Project license
โ”œโ”€โ”€ MANIFEST.in      # Describe the files included or not included in built package
โ”œโ”€โ”€ CHANGELOG.md     # Project changelog
โ”œโ”€โ”€ README.md        # Project description
โ”œโ”€โ”€ requirements.txt # Project dependency
โ”œโ”€โ”€ ruff.toml        # Define rules for code style, code inspection, and import management
โ”œโ”€โ”€ packaging.sh     # Package building script
โ”œโ”€โ”€ check_meta.sh    # Packaging metadata checking script
โ”œโ”€โ”€ setup.cfg        # Packaging configuration
โ””โ”€โ”€ setup.py         # Packaging script
๐’๐ญ๐š๐ง๐๐š๐ซ๐ ๐ฒ๐ž๐ญ ๐ก๐ข๐ ๐ก๐ฅ๐ฒ ๐œ๐ฎ๐ฌ๐ญ๐จ๐ฆ๐ข๐ณ๐š๐›๐ฅ๐ž
  • We standardize code sytle and quality with the wonderful Python linter and formatter: Ruff.
  • We standardize contributing pipeline with CONTRIBUTING.md to cut communication costs and boost development efficiency.
  • We offer ready-to-use templates for issue, pull requests(PR), and package publishing workflows, complete with modifications and usage instructions to help you customize them effectively.

๐Ÿ”จ Usage

Important

  • In demo below, we assume that your github ID is me and project name is my-project.
    โ—๏ธโ—๏ธโ—๏ธ Remember to replace them with your own ID and project name when using โ—๏ธโ—๏ธโ—๏ธ

  • This template uses setup.cfg to manage all metadata by default. While pyproject.toml is an officially recommended alternative, I find it more complicated, so I prefer setup.cfg. But if you really want to use pyproject.toml, please replace the setup.cfg with pyproject.toml below. Of course, you can download it directly here.

    • ๐š™๐šข๐š™๐š›๐š˜๐š“๐šŽ๐šŒ๐š.๐š๐š˜๐š–๐š•
      # refer to https://packaging.python.org/en/latest/guides/writing-pyproject-toml
      # See https://docs.astral.sh/ruff/settings for configuring ruff
      
      [build-system]  # define build backend and dependencies needed to build your project
      requires = ["setuptools>=66.0", "cython", "wheel", "isort", "ruff"]           # dependencies needed to build your project
      build-backend = "setuptools.build_meta"                             # build backend
      
      [project] # define metadata of your project
      
      # ---------------- Dynamic info ----------------
      dynamic = ["version","dependencies"]                                # dynamic info will be filled in by the build backend
      
      # ---------------- Basic info ----------------
      name = "your-package"                                               # package name
      authors = [
        { name="your-name", email="[email protected]" }, 
      ]
      maintainers = [
        { name="your-name", email="[email protected]" }, 
      ]
      description = "Package test"                             # one-line description of your project
      readme = {file = "README.md", content-type = "text/markdown"}       # specify README file
      
      # ---------------- Dependency info ----------------
      requires-python = ">=3.7"                                           # Python version requirement
      
      # ---------------- Other ----------------
      keywords = ["A","B","c"]      # keywords of your project, will help to suggest your project when people search for these keywords.
      classifiers = [               # Trove classifiers, Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
        "Topic :: Software Development :: Build Tools",
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.7",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
        "Programming Language :: Python :: 3.11",
        "Programming Language :: Python :: 3.12",
      ]
      
      # ---------------- Optional dependency ----------------
      [project.optional-dependencies] 
      docs = ["sphinx>=7.0.0"]
      
      test = [
        "pytest", 
        "pytest-sugar"]
      
      cli = [
        "rich",
        "click",
      ]
      
      # Install a command as part of your package
      [project.gui-scripts]                           # use [project.gui-scripts] to compatiable with differernt system   
      your-package = "your-package.cli:app"           # command = package:func
      
      
      # URLs associated with your project
      [project.urls]
      Homepage = "https://github.com/your-name/your-package"                    
      Repository = "https://github.com/your-name/your-package.git" 
      Issues = "https://github.com/your-name/your-package/issues" 
      Changelog = "https://github.com/your-name/your-package/blob/master/CHANGELOG.md"
      
      [tool.setuptools.dynamic]
      version = {attr = "your-package.__version__"}  # automatically obtain the value by `my_package.__version__`.
      dependencies = {file = ["requirements.txt", "requirement.txt", > "requirement"]}
      
      # -------------------------------- Tools Setting --------------------------------
      [tool.setuptools]
      license-files = ['LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*']  # specify License files
      
      [tool.setuptools.packages]
      find = {}  # Scan the project directory with the default parameters
      
      [tool.ruff]
      # Allow lines to be as long as 120.
      line-length = 120
      
      [tool.ruff.format]
      # Enable reformatting of code snippets in docstrings.
      docstring-code-format = true
      
      [tool.ruff.lint]
      # Skip unused variable rules
      ignore = [
          "ANN101",  # Missing type annotation for `self` in method
          "ANN102",  # Missing type annotation for `cls` in classmethod
          "ANN401",  # Dynamically typed expressions (typing.Any) are disallowed
          "C901",    # function is too complex (12 > 10)
          "COM812",  # Trailing comma missing
          "D",       # Docstring rules
          "EM101",   # Exception must not use a string literal, assign to variable first
          "EM102",   # Exception must not use an f-string literal, assign to variable first
          "ERA001",  # Found commented-out code
          "FBT001",  # Boolean positional arg in function definition
          "FBT002",  # Boolean default value in function definition
          "FBT003",  # Boolean positional value in function call
          "FIX002",  # Line contains TODO
          "ISC001",  # Isort
          "PLR0911", # Too many return statements (11 > 6)
          "PLR2004", # Magic value used in comparison, consider replacing 2 with a constant variable
          "PLR0912", # Too many branches
          "PLR0913", # Too many arguments to function call
          "PLR0915", # Too many statements
          "S101",    # Use of `assert` detected
          "S311",    # Standard pseudo-random generators are not suitable for cryptographic purposes
          "T201",    # print() found
          "T203",    # pprint() found
          "TD002",   # Missing author in TODO; try: `# TODO(<author_name>): ...`
          "TD003",   # Missing issue link on the line following this TODO
          "TD005",   # Missing issue description after `TODO`
          "TRY003",  # Avoid specifying long messages outside the exception class
          "PLW2901", # `for` loop variable `name` overwritten by assignment target
          "SLF001",  # Private member accessed: `_modules`
      ]
      
      [tool.ruff.lint.isort]
      length-sort = true                              # sort imports by their string length
      combine-as-imports = true                       # combines as imports on the same line
      known-first-party = ["your-package"]
      lines-after-imports = 1                         # Use a single line after each import block.
      single-line-exclusions = ["os", "json", "re"]   # modules to exclude from the single line rule
  1. ๐Ÿš€ ๐‚๐ซ๐ž๐š๐ญ๐ž ๐ฒ๐จ๐ฎ๐ซ ๐ซ๐ž๐ฉ๐จ

    Press the Use this template button next to star button at the top of this page,
    so as to use this repo as a template to create your repo.

  2. ๐Ÿ“ฅ ๐‚๐ฅ๐จ๐ง๐ž ๐ฒ๐จ๐ฎ๐ซ ๐ซ๐ž๐ฉ๐จ ๐ญ๐จ ๐ฅ๐จ๐œ๐š๐ฅ ๐ฆ๐š๐œ๐ก๐ข๐ง๐ž

    Find the newly created repo on your GitHub repositories page.
    Pull it to your machine with git clone.

    # replace 'me' with your github ID, 
    # 'my-project' with your project name, 
    # and `MYPROJECT` with your local project folder name
    git clone https://github.com/me/my-project MYPROJECT
  3. โœ๏ธ ๐‘๐ž๐ง๐š๐ฆ๐ž ๐ฉ๐ซ๐จ๐ฃ๐ž๐œ๐ญ ๐Ÿ๐จ๐ฅ๐๐ž๐ซ
    cd MYPROJECT
    
    # replace 'my-project' with your project name
    git mv package-name my-project
    ๐˜ฏ๐˜ฐ๐˜ธ ๐˜บ๐˜ฐ๐˜ถ๐˜ณ ๐˜ฑ๐˜ณ๐˜ฐ๐˜ซ๐˜ฆ๐˜ค๐˜ต ๐˜ด๐˜ต๐˜ณ๐˜ถ๐˜ค๐˜ต๐˜ถ๐˜ณ๐˜ฆ ๐˜ด๐˜ฉ๐˜ฐ๐˜ถ๐˜ญ๐˜ฅ ๐˜ฃ๐˜ฆ ๐˜ญ๐˜ช๐˜ฌ๐˜ฆ ๐˜ต๐˜ฉ๐˜ช๐˜ด
    # Note: 
    # the directory structure below neglects the `.github` dir
    
    MYPROJECT/
    โ”œโ”€โ”€ tests/ 
    โ”‚   โ””โ”€โ”€ README.md     
    |      
    โ”œโ”€โ”€ docs/   
    โ”‚   โ””โ”€โ”€ README.md    
    |            
    โ”œโ”€โ”€ examples/  
    โ”‚   โ””โ”€โ”€ demo.ipynb    
    |         
    โ”œโ”€โ”€ my-project/    
    โ”‚   โ”œโ”€โ”€ core.py      
    โ”‚   โ””โ”€โ”€ __init__.py   
    |
    โ”œโ”€โ”€ .gitignore   
    โ”œโ”€โ”€ LICENSE          
    โ”œโ”€โ”€ MANIFEST.in     
    โ”œโ”€โ”€ CHANGELOG.md     
    โ”œโ”€โ”€ README.md        
    โ”œโ”€โ”€ requirements.txt 
    โ”œโ”€โ”€ ruff.toml       
    โ”œโ”€โ”€ packaging.sh     
    โ”œโ”€โ”€ check_meta.sh    
    โ”œโ”€โ”€ setup.cfg        
    โ””โ”€โ”€ setup.py         
    
  4. ๐Ÿ“„ ๐Œ๐จ๐๐ข๐Ÿ๐ฒ ๐ญ๐ก๐ž ๐Ÿ๐จ๐ฅ๐ฅ๐จ๐ฐ๐ข๐ง๐  ๐Ÿ๐ข๐ฅ๐ž๐ฌ
    โ‘  ๐šœ๐šŽ๐š๐šž๐š™.๐šŒ๐š๐š / ๐š™๐šข๐š™๐š›๐š˜๐š“๐šŽ๐šŒ๐š.๐š๐š˜๐š–๐š• (๐š–๐š˜๐šœ๐š ๐š’๐š–๐š™๐š˜๐š›๐š๐šŠ๐š—๐š)

    ๐Ÿ’ก Tips

    โ€ข If your README is in rst format, you need to replace "text/markdown" with "text/x-rst" in long_description_content_type(setup.cfg) or readme(pyproject.toml).

    โ€ข If you want to create a CLI command for your package, enable [options.entry_points] option in setup.cfg or [project.gui-scripts] in pyproject.toml. See more here.

    โ€ข If you want more configuration, refer to keywords of setup.cfg or keywords of pyproject.toml

    Look for the following variables in setup.cfg and modify as per comments.

    Basic Requirement related Package structure related
    name python_requires packages
    version install_requires include_package_data
    author exclude
    author_email [options.extras_require]
    description
    long_description
    url
    keywords
    license
    classifiers

    If you are using pyproject.toml, you may need to replace your-package with my-package in the file we provided first, then check out and modify following variables.

    Basic Requirement related Package structure related
    name requires find
    version requires-python
    authors [project.optional-dependencies]
    maintainers
    description
    readme
    [project.urls]
    keywords
    classifiers
    โ‘ก ๐š–๐šข-๐š™๐š›๐š˜๐š“๐šŽ๐šŒ๐š/__๐š’๐š—๐š’๐š__.๐š™๐šข
    • line 2: <your-name> โ†’ me, replace with your github ID
    • replace <license-name> with your license name
    • replace <full_text-url-of-license-terms> with your license url, attain it from choosealicense.com
    • line 8: 0.1.0 โ†’ 0.0.1, replace with your project initial version
    โ‘ข ๐š›๐šž๐š๐š.๐š๐š˜๐š–๐š•

    โ€ข Here show the common change of ruff.toml
    โ€ข With comments in the file, you can modify everything as needed.
    โ€ข If you want more configuration, refer to Ruff document

    • line 3: target-version = "py37" โ†’ "py310", replace with your target python
    • line 46: known-first-party = ["<your_package_name>"] โ†’ ["my-project"], replace with your project name
    โ‘ฃ ๐š›๐šŽ๐šš๐šž๐š’๐š›๐šŽ๐š–๐šŽ๐š—๐š๐šœ.๐š๐šก๐š

    Here is an example, change it with the concrete dependencies that your project actually uses.

    setuptools
    isort
    ruff
    opencv-python
    tqdm
    
    โ‘ค ๐š๐™ด๐™ฐ๐™ณ๐™ผ๐™ด.๐š–๐š

    Here is an example, change it with your project description.

    # ๐Ÿง my-project
    
    ![Static Badge](https://img.shields.io/badge/Version-v0.0.1-green)
    
    ## ๐Ÿ‘‹ Introduction
    
    This is my first Python package called `my-project`.
    
    ## ๐Ÿ“ฆ Getting Started
    
    Install the package with pip: `pip install my-project`
    
    ## ๐Ÿ“„ License
    
    This project is licensed under the MIT License, 
    see the [LICENSE.md](LICENSE.md) for details
    
    ## ๐Ÿ’– Acknowledge
    
    Thanks for John for his help.
    โ‘ฅ ๐™ป๐š’๐šŒ๐šŽ๐š—๐šœ๐šŽ

    Default license is MIT, you can change it to other.
    See https://choosealicense.com/licenses/

    line 3: Copyright (c) <YEAR> <COPYRIGHT HOLDER>
    โ†“
    line 3: Copyright (c) 2024 me
    
    โ‘ฆ .๐š๐š’๐š๐š‘๐šž๐š‹/๐š ๐š˜๐š›๐š”๐š๐š•๐š˜๐š ๐šœ/๐š™๐šž๐š‹๐š•๐š’๐šœ๐š‘_๐š›๐šŽ๐š•๐šŽ๐šŠ๐šœ๐šŽ.๐šข๐š–๐š•

    โ€ข Change this file to use Github Actions for package publication.
    โ€ข If you want to change the preset workflow, see the CI/CD via Github Action ๐Ÿค– section below, or refer to Github Actions document

    • <package-name> โ†’ my-project
  5. ๐Ÿ‘จโ€๐Ÿ’ป ๐ƒ๐ž๐ฏ๐ž๐ฅ๐จ๐ฉ ๐ฒ๐จ๐ฎ๐ซ ๐ฉ๐ซ๐จ๐ฃ๐ž๐œ๐ญ

    ๐Ÿ’ก Tips
    โ€ข Cross-module imports can be made via .module-name or my-project.module-name in each module file.

    โ€ข You can test your code using python -m my-project.<module-name> with working directory in MYPROJECT.

    โ€ข To develop a command-line tool, add __main__.py in my-project folder. It defines logit when typing my-project in terminal. See more here

    Fill your logit into my-project folder.

  6. ๐Ÿ—ณ ๐๐ฎ๐ข๐ฅ๐ ๐๐ข๐ฌ๐ญ๐ซ๐ข๐›๐ฎ๐ญ๐ข๐จ๐ง ๐ฉ๐š๐œ๐ค๐š๐ ๐ž๐ฌ

    This step will generate .tar.gz source distribution file and .whl built distribution in a new folder called dist .

    # pwd: .../MYPROJECT
    chmod +x packaging.sh
    
    # Assume you are using anaconda to manage your python environment
    ./packaging.sh
    
    # Otherwise, activate your environment and execute following command
    python -m build -v -n .
  7. ๐Ÿ” ๐•๐š๐ฅ๐ข๐๐š๐ญ๐ž ๐ฉ๐š๐œ๐ค๐š๐ ๐ž

    โ‘ . ๐–ต๐–บ๐—…๐—‚๐–ฝ๐–บ๐—๐–พ ๐–ฝ๐—‚๐—Œ๐—๐—‹๐—‚๐–ป๐—Ž๐—๐—‚๐—ˆ๐—‡ ๐—†๐–พ๐—๐–บ๐–ฝ๐–บ๐—๐–บ

    # pwd: .../MYPROJECT
    pip install twine
    
    chmod +x check_meta.sh
    ./check_meta.sh

    โ‘ก. ๐–ต๐–บ๐—…๐—‚๐–ฝ๐–บ๐—๐–พ ๐–ฌ๐– ๐–ญ๐–จ๐–ฅ๐–ค๐–ฒ๐–ณ.๐—‚๐—‡ ๐—‚๐–ฟ ๐—’๐—ˆ๐—Ž ๐—๐–บ๐—๐–พ ๐—๐—๐—‚๐—Œ ๐–ฟ๐—‚๐—…๐–พ.

    # pwd: .../MYPROJECT
    pip install check-manifest
    
    # command below will automatically add missing file patterns to MANIFEST.in.
    check-manifest -u -v

    โ‘ข. (๐–ฎ๐—‰๐—๐—‚๐—ˆ๐—‡) ๐–ต๐–บ๐—…๐—‚๐–ฝ๐–บ๐—๐–พ ๐—‰๐–บ๐–ผ๐—„๐–บ๐—€๐–พ ๐–ฟ๐—Ž๐—‡๐–ผ๐—๐—‚๐—ˆ๐—‡๐—Œ

    # pwd: .../MYPROJECT
    pip install dist/*.whl
    
    # then test your package to see whether it works well.
    # this is suggested if you have create a CLI tool for your package.
  8. ๐Ÿ“ข ๐๐ฎ๐›๐ฅ๐ข๐ฌ๐ก ๐ฉ๐š๐œ๐ค๐š๐ ๐ž

    โ€ข This step will upload your package to PyPI or TestPyPI.
    โ€ข So firstly, you need to register an account with PyPI or TestPyPI.
    โ€ข Also, don't forget to generate a token for uploading your package. See more here.

    ๐Ÿ“‹ ๐–ฒ๐—Ž๐—€๐—€๐–พ๐—Œ๐—๐—‚๐—ˆ๐—‡
    You likely have many commits to PyPI or TestPyPI to familiarize yourself with the publishing operation. In this case, you can maintain a forged PyPI server locally, see the ๐Ÿงฐ Tools Recommended -> pypi-server section below.

    # pwd: .../MYPROJECT
    
    # (Option but strongly recommended) upload to testpypi firstly to see if anywhere wrong
    twine upload --repository testpypi dist/* 
    
    # upload to pypi
    # then everyone can install your package via `pip install my-project`
    twine upload --repository pypi dist/* 

    After executing command above, you will be asked to enter your account token.

    • Sure, you can paste your token in terminal to go through the process.

    • But if you are tired of doing this, you can use .pypirc and keyring to automatically access your token whenever needed. Follow the step in the configure .pypirc and keyring ๐Ÿ” section below.


๐Ÿฅณ ๐—–๐—ผ๐—ป๐—ด๐—ฟ๐—ฎ๐˜๐˜‚๐—น๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€!
โ€ข You have successfully published your package to PyPI.
โ€ข Now everyone can install it via pip install my-project
โ€ข To update your package to a new version, you have two choices:
โ‘  Manually update: repeat steps 5 to 8 above.
โ‘ก CI/CD workflow(recommended): see the CI/CD via Github Action ๐Ÿค– section below.

๐Ÿงฐ Tools Recommended

โ…  ๐š™๐šข๐š™๐š’-๐šœ๐šŽ๐š›๐šŸ๐šŽ๐š› ๐Ÿ–ฅ๏ธ

โ€ข What is it: A simple PyPI server for local use.
โ€ข Highly recommended if you are testing your CI/CD workflow.

You likely have many commits to PyPI or TestPyPI to familiarize yourself with publishing process. Then there exists two problems:

โ€ข TestPyPI / PyPI project size limit: many commits can exceed project size limit.

โ€ข Using TestPyPI as the index of pip install is not always reliable: especially when your package depends on some packages that are only available on PyPI but not on TestPyPI.

For example, if your package mp-project depends on ruff, then pip install mp-project -i https://test.pypi.org/simple will fail with ResolutionImpossible or Package not found in the process of finding and downloading ruff, cause ruff is only available on PyPI.

To solve these problems and fully imitate the bahvior of normal pip install using PyPI index. You can deploy a local PyPI server with pypi-server.

Here is a quick guide to get started, please check pypiserver's repo for more details.

pip install pypiserver 

mkdir Path/to/store/packages  # path to store distribution packages

pypi-server run \
-i 0.0.0.0 \
-p <port> \                  # specify a port to listen
<path-to-store>/.pypiserver_pkgs\
-a . -P . &                  # disable authentication for intranet use

cat >~/.pypirc<<EOF          # add local server to .pypirc
[distutils]
index-servers =
    pypi
    testpypi
    local

[pypi]
repository: https://upload.pypi.org/legacy/

[testpypi]
repository: https://test.pypi.org/legacy/

[local]
    repository: http://0.0.0.0:7418
    username: none          # random string, not important
    password: none          # random string, not important
EOF

OK, then we can use commands below to upload and install packages:

# pwd: .../package project dir

# upload package to local server
twine upload --repository local dist/*

# install package from local server
pip install <package> \
--trusted-host \
--extra-index-url http://0.0.0.0:<port>/simple/ 

โ—๏ธโ—๏ธโ—๏ธ If you want to close the server, using kill -9 "$(pgrep pypi-server)".

โ…ก ๐–ผ๐—ˆ๐—‡๐–ฟ๐—‚๐—€๐—Ž๐—‹๐–พ .๐š™๐šข๐š™๐š’๐š›๐šŒ ๐–บ๐—‡๐–ฝ ๐š”๐šŽ๐šข๐š›๐š’๐š—๐š ๐Ÿ”
  1. Configure keyring first

    pip install keyring keyrings.alt
    
    # if you are on Linux, execute commands below additionally.
    cat >"$(keyring diagnose | grep "config path:" | cut -d' ' -f3)"<<EOF
    [backend]
    default-keyring=keyrings.alt.file.PlaintextKeyring
    EOF
    
    # encrypt your pypi token 
    ## pypi
    keyring set https://upload.pypi.org/legacy/ __token__
    
    ## enter your pypi token when prompted
    
    # verify that the encrypted token has been stored
    keyring get https://upload.pypi.org/legacy/ __token__ 
    
    # ------------------------ same for testpypi ------------------------
    
    ## testpypi
    keyring set https://test.pypi.org/legacy/ __token__
    
    ## enter your pypi token when prompted
    
    # verify that the encrypted token has been stored
    keyring get https://test.pypi.org/legacy/ __token__
  2. Configure .pypirc

    # refer to https://packaging.python.org/en/latest/specifications/pypirc/
    cat >~/.pypirc<<EOF
    [distutils]
    index-servers =
        pypi
        testpypi
    
    [pypi]
    repository = https://upload.pypi.org/legacy/
    
    [testpypi]
    repository = https://test.pypi.org/legacy/
    EOF
    
    chmod 600 ~/.pypirc
  3. At this point, there is no need to verify your token manually when you upload packages via twine upload

๐Ÿ—ƒ Project Management

This section emphasizes the effective management of your project on GitHub.

โ…  ๐’๐ญ๐š๐ง๐๐š๐ซ๐๐ข๐ณ๐ž๐ ๐œ๐จ๐ง๐ญ๐ซ๐ข๐›๐ฎ๐ญ๐ข๐จ๐ง ๐ฉ๐ซ๐จ๐œ๐ž๐ฌ๐ฌ ๐Ÿ’ผ

Standardizing project participation cuts communication costs and boosts development efficiency. This mainly focus on the files below:

  1. .github/CONTRIBUTING.md : guide other to make contribution to your project. To change it, refer to link.

  2. .github/ISSUE_TEMPLATE : standardize the format of issue reporting. Composed of

    ๐Ÿ’ก Tips
    โ€ข Open the Issue page in this repo,to see what the template looks like.
    โ€ข If you are to change it, refer to link1, link2 and link3.

  3. .github/PULL_REQUEST_TEMPLATE.md : standardize the format of Pull Request. To change it, refer to link.

โ…ก ๐‚๐ˆ/๐‚๐ƒ ๐ฏ๐ข๐š ๐†๐ข๐ญ๐ก๐ฎ๐› ๐€๐œ๐ญ๐ข๐จ๐ง ๐Ÿค–

โš ๏ธโš ๏ธโš ๏ธ
โ€ข Due to the need of publishing to PyPI and TestPypi, trusted publishers of two platform needs to be configured first before use. Following tutorial 1 and tutorial 2 to make it.

โ€ข NOTE: The Environment name item in configuration should be the same as what you specify in the workflow file.

For example, in the provided publish_release.yml, the Environment name is pypi in PyPI platform, cause we specify it in job Publish-PyPI.environment.name.

  • By creating a .yml file under the .github/workflows/ directory, CI/CD support for the project can be achieved.

  • In this template repo, the automation of steps 6 to 8 in ๐Ÿ”จ Usage section is implemented. Once a push with a tag is made and the tag matches a template of the form v*.*.*, events below will happen:

    1. Build distribution packages, i.e., .tar.gz and .whl files
    2. Verify meta information of the distribution packages
    3. Release distribution packages to PyPI and TestPyPI, respectively
    4. Generate release according to tag name and CHANGELOG.md
    5. Upload the distribution package to the generated release.
  • If you are to change the task flows, please see Github Actions document for more details.

โ—๏ธโ—๏ธโ—๏ธ
If you want to disable the CI/CD feature, there are two options:
โ€ข delete the .github/workflows/ directory
โ€ข do Settings -> Actions -> General -> Disable actions in project setting.

๐Ÿ“‘ To Do

  • Add full pipeline of package development, from project preparation to maintaining.
  • Add CI/CD support, such as GitHub Actions
  • Add pyproject.toml support
  • Add linter

๐Ÿ‘€ See More

๐Ÿงพ License

This is free and unencumbered software released into the public domain. Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

About

๐Ÿ“ฆ Project template for self-developed Python package

Resources

License

Stars

Watchers

Forks