Add pyproject.toml, and stop calling setup.py directly (#629)

**Issue:**

For a few years now, we've been seen the occasional deprecation warning about setup.py. Including:

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer or other
    standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

and

    ********************************************************************************
    Ensure that any custom bdist_wheel implementation is a subclass of
    setuptools.command.bdist_wheel.bdist_wheel.

    By 2025-Oct-15, you need to update your project and remove deprecated calls
    or your builds will no longer be supported.

    See pypa/wheel#631 for details.
    ********************************************************************************

2025-Oct is on its way, so let's finally do something about it...

**Description of changes:**

Follow advice from ["Is setup.py deprecated?"](https://packaging.python.org/en/latest/discussions/setup-py-deprecated/) page at packaging.python.org

tldr; having a setup.py file **is not** deprecated, but directly invoking `python setup.py` **is** deprecated.

- Add [pyproject.toml](https://packaging.python.org/en/latest/specifications/pyproject-toml/) file (the modern way to configure your package)
    - Move data from `setup.py` -> `pyproject.toml` wherever possible (can't move dynamic stuff, like build steps for C extension)
    - Move dev dependencies from `requirements-dev.txt` -> `pyproject.toml [project.optional-dependencies]` 
        - Now you call `pip install ".[dev]"`, instead of `pip install -r requirements-dev.txt`. The dev dependencies will be installed by the same command that builds and installs our project.
        - This change isn't 100% necessary, but seemed like the modern way to do things. It's one less step to set things up on your machine. However, you can't install dependencies without ALSO building/installing the package (we can roll this back if we end up hating it)
- Use `python3 -m build` instead of `python3 setup.py sdist bdist_wheel` (the modern way to build a wheel for distribution)
    - Sadly, [build](https://pypi.org/project/build/) is a non-core package, so might need to be pip installed before using. Fortunately it's been [preinstalled on manylinux images since 2021](https://github.com/pypa/manylinux/blob/6b97f150fc9ede1cb4f25de84a939538a1019a58/docker/build_scripts/requirements3.8.txt#L7), so our Linux release process will be OK.
    - I updated our MacOS and Windows machines to have `build` installed for all python versions, so our release process should be OK.
- STOP calling `pip install --upgrade setuptools` before `pip install .`
    - Modern pip installs the "build dependencies" declared in `pyproject.toml` (setuptools & wheel) in a special isolated environment before it builds our project. We declared `setuptools>=75.3.1`, so we're guaranteed it will get installed, and be a recent version.

Author: graebm

.builder/actions/aws_crt_python.py

@@ -31,10 +31,10 @@ def run(self, env):
         env.shell.setenv('AWS_TEST_S3', '1')
 
         actions = [
+            [self.python, '--version'],
             [self.python, '-m', 'pip', 'install', '--upgrade', 'pip'],
-            [self.python, '-m', 'pip', 'install', '--upgrade', '--requirement', 'requirements-dev.txt'],
             Builder.SetupCrossCICrtEnvironment(),
-            [self.python, '-m', 'pip', 'install', '--verbose', '.'],
+            [self.python, '-m', 'pip', 'install', '--verbose', '.[dev]'],
             # "--failfast" because, given how our leak-detection in tests currently works,
             # once one test fails all the rest usually fail too.
             [self.python, '-m', 'unittest', 'discover', '--verbose', '--failfast'],

.github/workflows/ci.yml

@@ -337,8 +337,7 @@ jobs:
           submodules: true
       - name: Run tests
         run: |
-          python3 -m pip install --upgrade --requirement requirements-dev.txt
-          python3 -m pip install . --verbose
+          python3 -m pip install ".[dev]" --verbose
           python3 -m unittest discover --failfast --verbose
 
   package-source:
@@ -356,22 +355,23 @@ jobs:
           submodules: true
       - name: Package source + install
         run: |
-          python3 setup.py sdist
+          python3 -m pip install build
+          python3 -m build --sdist
           cd dist
           python3 -m pip install -v awscrt-1.0.0.dev0.tar.gz
           python3 -c "import awscrt.io"
 
   # check that docs can still build
   check-docs:
-    runs-on: ubuntu-22.04 # latest
+    runs-on: ubuntu-22.04 #  getting errors from sphinx using ubuntu-24.04, so stuck on 22.04 till we diagnose it
     steps:
       - uses: actions/checkout@v4
         with:
           submodules: true
       - name: Check docs
         run: |
-          python3 -m pip install -r requirements-dev.txt
-          python3 -m pip install --verbose .
+          python3 -m pip install --upgrade pip
+          python3 -m pip install --verbose ".[dev]"
           ./scripts/make-docs.py
 
   check-submodules:

.github/workflows/docs.yml

@@ -9,7 +9,7 @@ on:
 
 jobs:
   update-docs-branch:
-    runs-on: ubuntu-22.04 # latest
+    runs-on: ubuntu-22.04 # getting errors from sphinx using ubuntu-24.04, so stuck on 22.04 till we diagnose it
     permissions:
       contents: write # allow push
     steps:
@@ -20,8 +20,8 @@ jobs:
 
       - name: Update docs branch
         run: |
-          python3 -m pip install -r requirements-dev.txt
-          python3 -m pip install --verbose .
+          python3 -m pip install --upgrade pip
+          python3 -m pip install --verbose ".[dev]"
           ./scripts/make-docs.py
 
       - name: Commit

.github/workflows/lint.yml

@@ -7,28 +7,19 @@ on:
       - 'docs'
 
 jobs:
-  clang-format:
-
-    runs-on: ubuntu-24.04 # latest
-
-    steps:
-    - name: Checkout Sources
-      uses: actions/checkout@v4
-
-    - name: clang-format lint
-      run: |
-        ./format-check.py
-
-  autopep8:
+  lint:
     runs-on: ubuntu-24.04 # latest
 
     steps:
     - name: Checkout Source
       uses: actions/checkout@v4
 
-    - name: Build and Test
+    - name: Format and Check Diff
+      # run formatting script, which edits files in place
+      # then do `git diff` to show what changed (if anything)
+      # then do `git diff --quiet` to fail if anything changed
       run: |
-        python3 -m pip install --upgrade setuptools
-        python3 -m pip install --upgrade wheel
         python3 -m pip install --upgrade autopep8
-        python3 -m autopep8 --exit-code --diff --recursive awscrt test .builder setup.py
+        python3 scripts/format-all.py
+        git diff
+        git diff --quiet

.lgtm.yml

@@ -32,7 +32,7 @@ git submodule update --init
 python3 -m pip install .
 ```
 
-To use from your Python application, declare `awscrt` as a dependency in your `setup.py` file.
+To use from your Python application, declare `awscrt` as a dependency.
 
 ### OpenSSL and LibCrypto (Unix only)
 

README.md

@@ -12,9 +12,9 @@ phases:
   build:
     commands:
       - echo Build started on `date`
-      - /opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
+      - /opt/python/cp38-cp38/bin/python -m build
       - auditwheel repair --plat manylinux1_x86_64 dist/awscrt-*cp38-cp38-linux_x86_64.whl
-      - /opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
+      - /opt/python/cp39-cp39/bin/python -m build
       - auditwheel repair --plat manylinux1_x86_64 dist/awscrt-*cp39-cp39-linux_x86_64.whl
       # python 3.9 is the last version manylinux1 will ever receive
       - cp -r wheelhouse ../dist

codebuild/cd/manylinux-x64-build.yml

@@ -12,9 +12,9 @@ phases:
   build:
     commands:
       - echo Build started on `date`
-      - /opt/python/cp38-cp38/bin/python setup.py sdist bdist_wheel
+      - /opt/python/cp38-cp38/bin/python -m build
       - auditwheel repair --plat manylinux1_i686 dist/awscrt-*cp38-cp38-linux_i686.whl
-      - /opt/python/cp39-cp39/bin/python setup.py sdist bdist_wheel
+      - /opt/python/cp39-cp39/bin/python -m build
       - auditwheel repair --plat manylinux1_i686 dist/awscrt-*cp39-cp39-linux_i686.whl
       # python 3.9 is the last version manylinux1 will ever receive
       - cp -r wheelhouse ../dist

codebuild/cd/manylinux-x86-build.yml

@@ -6,7 +6,7 @@ phases:
       - sudo apt-get update -y
       - sudo apt-get install python3 python3-pip -y
       - python3 -m pip install --user --upgrade pip
-      - python3 -m pip install --user --upgrade twine setuptools wheel boto3 PyOpenSSL six
+      - python3 -m pip install --user --upgrade twine boto3
   pre_build:
     commands:
       - export CC=gcc

codebuild/cd/publish_to_prod_pypi.yml

@@ -6,7 +6,7 @@ phases:
       - sudo apt-get update -y
       - sudo apt-get install python3 python3-pip -y
       - python3 -m pip install --user --upgrade pip
-      - python3 -m pip install --user --upgrade twine setuptools wheel boto3 PyOpenSSL six
+      - python3 -m pip install --user --upgrade twine boto3
   pre_build:
     commands:
       - export CC=gcc