Skip to content

Commit 182f671

Browse files
florathflorath
committed
feat: Implement comprehensive PyPI release pipeline with safety checks
This addresses issue #10 by implementing and testing the complete PyPI release workflow with extensive safety measures and documentation. Changes: CI/CD Pipeline (.github/workflows/ci.yml): - Add test-pypi-publish job for automated TestPyPI releases on main branch - Implement comprehensive version validation (PEP 440 format checking) - Add version-tag matching validation for production releases - Check for duplicate versions on PyPI before publishing - Validate package metadata completeness - Add local installation testing from built packages - Test installation from TestPyPI after upload - Enhance publish job with strict safety checks and verification - Add post-publication verification with PyPI indexing wait Package Configuration (pyproject.toml): - Add release optional dependencies (build, twine, packaging, requests) - Enables: pip install -e .[release] Tooling (scripts/bump_version.py): - New version management script with semantic versioning support - Supports: major, minor, patch, or specific version bumps - Automatic pyproject.toml updates and git commit creation - Optional git tag creation with --tag flag - Safety checks for clean working directory - Comprehensive help and usage examples Documentation (docs/RELEASE.md): - Quick-start guide for experienced maintainers - Comprehensive automated CI/CD overview - Versioning strategy and bump script usage - Troubleshooting common release issues - Appendix A: Initial setup (secrets, environments) - Appendix B: Manual testing procedures - Appendix C: Security best practices - Appendix D: Emergency manual release procedures Safety Features: - Manual approval required for production releases (via GitHub environments) - Automated duplicate version detection - Version format validation (PEP 440) - Git tag and pyproject.toml version matching - TestPyPI testing before production - Package metadata validation - Post-publication verification Testing: - Local build tested successfully (python -m build) - Package validation passed (twine check) - Installation from wheel tested in clean venv - All version validation scripts tested Resolves #10
1 parent 4a8fc6e commit 182f671

File tree

4 files changed

+795
-3
lines changed

4 files changed

+795
-3
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 253 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -480,15 +480,122 @@ jobs:
480480
# id: deployment
481481
# uses: actions/deploy-pages@v4
482482

483+
test-pypi-publish:
484+
name: Test PyPI Publishing (TestPyPI)
485+
runs-on: ubuntu-latest
486+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
487+
needs: [lint-and-type-check, test, build, integration-tests]
488+
environment: test-pypi
489+
490+
steps:
491+
- uses: actions/checkout@v4
492+
with:
493+
fetch-depth: 0 # Fetch all history for version validation
494+
495+
- name: Set up Python 3.11
496+
uses: actions/setup-python@v5
497+
with:
498+
python-version: "3.11"
499+
500+
- name: Install dependencies
501+
run: |
502+
python -m pip install --upgrade pip
503+
pip install build twine packaging
504+
505+
- name: Validate version format
506+
run: |
507+
python -c "
508+
import re
509+
import sys
510+
from pathlib import Path
511+
512+
# Read version from pyproject.toml
513+
content = Path('pyproject.toml').read_text()
514+
match = re.search(r'version = \"(.+?)\"', content)
515+
if not match:
516+
print('ERROR: Could not find version in pyproject.toml')
517+
sys.exit(1)
518+
519+
version = match.group(1)
520+
print(f'Found version: {version}')
521+
522+
# Validate version format (PEP 440)
523+
if not re.match(r'^\d+\.\d+\.\d+(?:\.(?:dev|alpha|beta|rc)\d+)?$', version):
524+
print(f'ERROR: Version {version} does not match PEP 440 format')
525+
print('Expected format: X.Y.Z or X.Y.Z.devN or X.Y.Z.alphaN, etc.')
526+
sys.exit(1)
527+
528+
print(f'✓ Version {version} is valid')
529+
"
530+
531+
- name: Build package
532+
run: |
533+
python -m build
534+
535+
- name: Check package
536+
run: |
537+
twine check dist/*
538+
539+
- name: Test installation from built package
540+
run: |
541+
# Create a temporary virtual environment
542+
python -m venv test_env
543+
source test_env/bin/activate
544+
545+
# Install the built wheel
546+
pip install dist/*.whl
547+
548+
# Test that the CLI is available
549+
aletheia-probe --help
550+
551+
# Cleanup
552+
deactivate
553+
rm -rf test_env
554+
555+
- name: Publish to TestPyPI
556+
if: success()
557+
env:
558+
TWINE_USERNAME: __token__
559+
TWINE_PASSWORD: ${{ secrets.TESTPYPI_API_TOKEN }}
560+
run: |
561+
echo "Publishing to TestPyPI..."
562+
twine upload --repository testpypi dist/* --verbose || echo "Note: Upload may fail if version already exists on TestPyPI"
563+
564+
- name: Test installation from TestPyPI
565+
if: success()
566+
run: |
567+
echo "Waiting 30 seconds for TestPyPI to process the upload..."
568+
sleep 30
569+
570+
# Create a fresh virtual environment
571+
python -m venv testpypi_env
572+
source testpypi_env/bin/activate
573+
574+
# Try to install from TestPyPI
575+
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ aletheia-probe || echo "Installation from TestPyPI may fail if package was just uploaded"
576+
577+
# Cleanup
578+
deactivate
579+
rm -rf testpypi_env
580+
581+
- name: Archive test artifacts
582+
uses: actions/upload-artifact@v4
583+
if: always()
584+
with:
585+
name: testpypi-artifacts
586+
path: dist/
587+
483588
publish:
484-
name: Publish to PyPI
589+
name: Publish to PyPI (Production)
485590
runs-on: ubuntu-latest
486591
if: github.event_name == 'release' && github.event.action == 'published'
487592
needs: [lint-and-type-check, test, build, integration-tests, cross-platform-integration]
488593
environment: production
489594

490595
steps:
491596
- uses: actions/checkout@v4
597+
with:
598+
fetch-depth: 0 # Fetch all history for version validation
492599

493600
- name: Set up Python 3.11
494601
uses: actions/setup-python@v5
@@ -498,18 +605,161 @@ jobs:
498605
- name: Install dependencies
499606
run: |
500607
python -m pip install --upgrade pip
501-
pip install build twine
608+
pip install build twine packaging requests
609+
610+
- name: Validate version matches git tag
611+
run: |
612+
python -c "
613+
import re
614+
import sys
615+
import os
616+
from pathlib import Path
617+
618+
# Read version from pyproject.toml
619+
content = Path('pyproject.toml').read_text()
620+
match = re.search(r'version = \"(.+?)\"', content)
621+
if not match:
622+
print('ERROR: Could not find version in pyproject.toml')
623+
sys.exit(1)
624+
625+
pyproject_version = match.group(1)
626+
print(f'Version in pyproject.toml: {pyproject_version}')
627+
628+
# Get git tag
629+
git_ref = os.environ.get('GITHUB_REF', '')
630+
if git_ref.startswith('refs/tags/'):
631+
git_tag = git_ref.replace('refs/tags/', '')
632+
# Remove 'v' prefix if present
633+
git_version = git_tag.lstrip('v')
634+
print(f'Git tag version: {git_version}')
635+
636+
if pyproject_version != git_version:
637+
print(f'ERROR: Version mismatch!')
638+
print(f' pyproject.toml: {pyproject_version}')
639+
print(f' git tag: {git_version}')
640+
sys.exit(1)
641+
642+
print(f'✓ Versions match: {pyproject_version}')
643+
else:
644+
print('WARNING: Not running from a tag, skipping tag validation')
645+
"
646+
647+
- name: Check if version exists on PyPI
648+
run: |
649+
python -c "
650+
import sys
651+
import re
652+
import requests
653+
from pathlib import Path
654+
655+
# Read version from pyproject.toml
656+
content = Path('pyproject.toml').read_text()
657+
match = re.search(r'version = \"(.+?)\"', content)
658+
version = match.group(1)
659+
660+
# Check PyPI
661+
response = requests.get(f'https://pypi.org/pypi/aletheia-probe/{version}/json')
662+
if response.status_code == 200:
663+
print(f'ERROR: Version {version} already exists on PyPI!')
664+
print(f'Please bump the version in pyproject.toml before releasing.')
665+
sys.exit(1)
666+
elif response.status_code == 404:
667+
print(f'✓ Version {version} does not exist on PyPI - safe to publish')
668+
else:
669+
print(f'WARNING: Could not verify version on PyPI (status {response.status_code})')
670+
print('Proceeding with caution...')
671+
"
672+
673+
- name: Validate package metadata
674+
run: |
675+
python -c "
676+
import sys
677+
from pathlib import Path
678+
import re
679+
680+
content = Path('pyproject.toml').read_text()
681+
682+
# Check required fields
683+
required = ['name', 'version', 'description', 'readme', 'license', 'authors']
684+
missing = []
685+
686+
for field in required:
687+
if field not in content:
688+
missing.append(field)
689+
690+
if missing:
691+
print(f'ERROR: Missing required fields in pyproject.toml: {missing}')
692+
sys.exit(1)
693+
694+
print('✓ All required metadata fields present')
695+
696+
# Validate URLs
697+
if 'Homepage' not in content:
698+
print('WARNING: Homepage URL not set')
699+
if 'Repository' not in content:
700+
print('WARNING: Repository URL not set')
701+
702+
print('✓ Package metadata validation passed')
703+
"
502704
503705
- name: Build package
504706
run: |
505707
python -m build
506708
709+
- name: Check package with twine
710+
run: |
711+
twine check dist/* --strict
712+
713+
- name: Display package contents
714+
run: |
715+
echo "📦 Package contents:"
716+
tar -tzf dist/*.tar.gz | head -30
717+
echo ""
718+
echo "📊 Package size:"
719+
du -h dist/*
720+
507721
- name: Publish to PyPI
508722
env:
509723
TWINE_USERNAME: __token__
510724
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
511725
run: |
512-
twine upload dist/*
726+
echo "🚀 Publishing to PyPI..."
727+
twine upload dist/* --verbose
728+
729+
- name: Verify published package
730+
run: |
731+
python -c "
732+
import sys
733+
import time
734+
import requests
735+
from pathlib import Path
736+
import re
737+
738+
# Read version from pyproject.toml
739+
content = Path('pyproject.toml').read_text()
740+
match = re.search(r'version = \"(.+?)\"', content)
741+
version = match.group(1)
742+
743+
print(f'Waiting for PyPI to index version {version}...')
744+
745+
# Wait up to 2 minutes for the package to appear
746+
for i in range(24):
747+
response = requests.get(f'https://pypi.org/pypi/aletheia-probe/{version}/json')
748+
if response.status_code == 200:
749+
print(f'✓ Package successfully published to PyPI!')
750+
print(f'🔗 https://pypi.org/project/aletheia-probe/{version}/')
751+
sys.exit(0)
752+
time.sleep(5)
753+
754+
print('WARNING: Package not yet visible on PyPI after 2 minutes')
755+
print('This may be normal - check manually at: https://pypi.org/project/aletheia-probe/')
756+
"
757+
758+
- name: Archive release artifacts
759+
uses: actions/upload-artifact@v4
760+
with:
761+
name: release-packages
762+
path: dist/
513763

514764
notify:
515765
name: Notify

0 commit comments

Comments
 (0)