Integrate pytest-jux into real projects and CI/CD pipelines
By the end of this tutorial, you will:
✅ Integrate pytest-jux into an existing Python project ✅ Configure multi-environment signing (dev, staging, production) ✅ Set up GitHub Actions CI/CD with automated signing ✅ Manage signing keys securely in CI/CD ✅ Implement report storage and caching strategies ✅ Debug signing issues in complex projects
Time Required: 30-45 minutes
Difficulty: Intermediate
Prerequisites:
- Completed First Signed Report tutorial
- Familiarity with pytest test suites
- Basic CI/CD knowledge (GitHub Actions or GitLab CI)
- Understanding of environment variables
This tutorial simulates integrating pytest-jux into a real-world Python project with:
- Multiple test files and fixtures
- Development, staging, and production environments
- GitHub Actions CI/CD pipeline
- Secure key management
- Report storage and archival
Scenario: Your team maintains a Python library and wants to publish signed test reports to verify code quality for downstream users.
For this tutorial, we'll create a realistic project structure.
# Create project directory
mkdir -p pytest-jux-integration-demo
cd pytest-jux-integration-demo
# Initialize git repository
git init
git config user.name "Your Name"
git config user.email "your.email@example.com"# Create project structure
mkdir -p src/mylib tests/unit tests/integration .github/workflows
# Create package files
cat > src/mylib/__init__.py << 'EOF'
"""My Library - A sample Python library."""
__version__ = "1.0.0"
def add(a: int, b: int) -> int:
"""Add two numbers."""
return a + b
def multiply(a: int, b: int) -> int:
"""Multiply two numbers."""
return a * b
def divide(a: float, b: float) -> float:
"""Divide two numbers."""
if b == 0:
raise ValueError("Cannot divide by zero")
return a / b
EOF
# Create unit tests
cat > tests/unit/test_math.py << 'EOF'
"""Unit tests for math operations."""
import pytest
from mylib import add, multiply, divide
def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
def test_multiply():
assert multiply(2, 3) == 6
assert multiply(-2, 3) == -6
def test_divide():
assert divide(6, 2) == 3.0
assert divide(1, 2) == 0.5
def test_divide_by_zero():
with pytest.raises(ValueError, match="Cannot divide by zero"):
divide(1, 0)
EOF
# Create integration tests
cat > tests/integration/test_workflows.py << 'EOF'
"""Integration tests for common workflows."""
from mylib import add, multiply, divide
def test_complex_calculation():
"""Test a complex calculation workflow."""
result = add(multiply(2, 3), divide(10, 2))
assert result == 11.0
def test_chain_operations():
"""Test chaining multiple operations."""
a = add(1, 2)
b = multiply(a, 3)
c = divide(b, 2)
assert c == 4.5
EOF
# Create pytest configuration
cat > pytest.ini << 'EOF'
[pytest]
minversion = 7.4
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
-v
--strict-markers
--junitxml=test-results/junit.xml
# pytest-jux configuration (will be environment-specific)
jux_environment = development
jux_storage_mode = auto
EOFcat > pyproject.toml << 'EOF'
[project]
name = "mylib"
version = "1.0.0"
description = "A sample Python library for pytest-jux integration tutorial"
requires-python = ">=3.11"
dependencies = []
[project.optional-dependencies]
dev = [
"pytest>=7.4",
"pytest-jux>=0.1.9",
"pytest-cov>=4.1",
]
[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"
EOF# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Install dependencies
pip install -e ".[dev]"✓ Checkpoint: Project structure created, dependencies installed
# Create configuration directory
mkdir -p config
# Development configuration
cat > config/development.toml << 'EOF'
[jux]
environment = "development"
storage_mode = "disabled" # Don't cache in dev
[jux.signing]
# Use ephemeral keys in development
private_key_path = "~/.ssh/jux/dev-key.pem"
certificate_path = "~/.ssh/jux/dev-cert.pem"
EOF
# Staging configuration
cat > config/staging.toml << 'EOF'
[jux]
environment = "staging"
storage_mode = "auto"
[jux.signing]
private_key_path = "~/.ssh/jux/staging-key.pem"
certificate_path = "~/.ssh/jux/staging-cert.pem"
EOF
# Production configuration
cat > config/production.toml << 'EOF'
[jux]
environment = "production"
storage_mode = "auto"
[jux.signing]
private_key_path = "~/.ssh/jux/prod-key.pem"
certificate_path = "~/.ssh/jux/prod-cert.pem"
EOF# Create keys directory
mkdir -p ~/.ssh/jux
# Generate development key
jux-keygen \
--type rsa \
--bits 2048 \
--output ~/.ssh/jux/dev-key.pem \
--cert \
--subject "CN=MyLib Development" \
--days-valid 90
# Generate staging key
jux-keygen \
--type rsa \
--bits 4096 \
--output ~/.ssh/jux/staging-key.pem \
--cert \
--subject "CN=MyLib Staging" \
--days-valid 180
# Generate production key
jux-keygen \
--type rsa \
--bits 4096 \
--output ~/.ssh/jux/prod-key.pem \
--cert \
--subject "CN=MyLib Production" \
--days-valid 365✓ Checkpoint: Multi-environment configuration created
# Run tests with development config
pytest \
--jux-config config/development.toml \
--jux-key ~/.ssh/jux/dev-key.pem \
--jux-cert ~/.ssh/jux/dev-cert.pemExpected Output:
======================== test session starts =========================
collected 6 items
tests/integration/test_workflows.py .. [ 33%]
tests/unit/test_math.py .... [100%]
======================== 6 passed in 0.15s ==========================
pytest-jux: Report signed and stored
Environment: development
Storage mode: disabled (not cached)
Signed report: test-results/junit.xml
# Run tests with staging config
pytest \
--jux-config config/staging.toml \
--jux-key ~/.ssh/jux/staging-key.pem \
--jux-cert ~/.ssh/jux/staging-cert.pemExpected Output:
======================== 6 passed in 0.15s ==========================
pytest-jux: Report signed and stored
Environment: staging
Hash: b2c3d4e5f6g7h8i9...
Signed report: ~/.local/share/pytest-jux/reports/b2c3d4...xml
Notice: Staging reports are cached (storage_mode = "auto"), development reports are not.
✓ Checkpoint: Multi-environment testing works correctly
# Create workflow file
cat > .github/workflows/test.yml << 'EOF'
name: Test with Signed Reports
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11", "3.12"]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
- name: Generate ephemeral signing key
run: |
mkdir -p ~/.ssh/jux
jux-keygen \
--type rsa \
--bits 4096 \
--output ~/.ssh/jux/ci-key.pem \
--cert \
--subject "CN=GitHub Actions CI" \
--days-valid 1
- name: Run tests with signing
env:
JUX_ENVIRONMENT: ci
JUX_KEY_PATH: ~/.ssh/jux/ci-key.pem
JUX_CERT_PATH: ~/.ssh/jux/ci-key.crt
JUX_STORAGE_MODE: disabled
run: |
pytest \
--junitxml=test-results/junit.xml \
--cov=src/mylib \
--cov-report=xml \
--cov-report=term
- name: Verify signed report
run: |
jux-verify \
-i test-results/junit.xml \
--cert ~/.ssh/jux/ci-key.crt
- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-py${{ matrix.python-version }}
path: |
test-results/
coverage.xml
- name: Upload signed reports
if: always()
uses: actions/upload-artifact@v4
with:
name: signed-reports-py${{ matrix.python-version }}
path: test-results/junit.xml
EOF# Add files to git
git add .
git commit -m "feat: Add pytest-jux integration with CI/CD"
# Create GitHub repository (if not exists)
# Then push:
# git remote add origin https://github.com/yourusername/pytest-jux-integration-demo.git
# git push -u origin main✓ Checkpoint: GitHub Actions workflow configured
Pros: No secrets to manage, keys rotate automatically Cons: Can't verify reports after CI run completes
# Already implemented in workflow above
- name: Generate ephemeral signing key
run: |
jux-keygen \
--type rsa \
--bits 4096 \
--output ~/.ssh/jux/ci-key.pem \
--certPros: Persistent keys, reports verifiable long-term Cons: Requires secret management
Setup:
- Generate production key locally:
jux-keygen \
--type rsa \
--bits 4096 \
--output prod-ci-key.pem \
--cert \
--subject "CN=MyLib CI Production"-
Add to GitHub Secrets:
- Go to: Repository → Settings → Secrets and variables → Actions
- Add secrets:
JUX_PRIVATE_KEY: Content ofprod-ci-key.pemJUX_CERTIFICATE: Content ofprod-ci-key.crt
-
Update workflow:
- name: Configure signing keys from secrets
run: |
mkdir -p ~/.ssh/jux
echo "${{ secrets.JUX_PRIVATE_KEY }}" > ~/.ssh/jux/ci-key.pem
echo "${{ secrets.JUX_CERTIFICATE }}" > ~/.ssh/jux/ci-key.crt
chmod 600 ~/.ssh/jux/ci-key.pem
chmod 644 ~/.ssh/jux/ci-key.crt✓ Checkpoint: Secure key management configured
# View cache statistics
jux-cache statsExpected Output:
Cache Statistics:
Total reports: 3
Total size: 42.5 KB
Oldest report: 2025-10-20T10:00:00Z
Newest report: 2025-10-20T14:30:00Z
Storage location: ~/.local/share/pytest-jux/
# List all cached reports
jux-cache listExpected Output:
Cached Reports (3 total):
b2c3d4e5f6g7h8i9...
Timestamp: 2025-10-20T14:30:00Z
Hostname: your-machine.local
Username: you
c3d4e5f6g7h8i9j0...
Timestamp: 2025-10-20T12:15:00Z
Hostname: your-machine.local
Username: you
# Remove reports older than 30 days
jux-cache clean --days 30Expected Output:
Cleaned up 0 reports older than 30 days
In GitHub Actions, reports are automatically archived as artifacts. Download them:
- Go to: Actions → Your workflow run
- Scroll to "Artifacts" section
- Download "signed-reports-py3.11"
✓ Checkpoint: Report storage and archival configured
Symptom:
Error: Private key not found: ~/.ssh/jux/ci-key.pem
Cause: Tilde expansion doesn't work in environment variables
Solution: Use absolute path or $HOME:
env:
JUX_KEY_PATH: $HOME/.ssh/jux/ci-key.pemSymptom: Tests pass but no signing message appears
Cause: Missing --junitxml option
Solution: Ensure pytest.ini has:
[pytest]
addopts = --junitxml=test-results/junit.xmlSymptom:
Error: Signature verification failed
Debug Steps:
- Check certificate matches key:
# Extract public key from private key
openssl rsa -in key.pem -pubout -out key-pub.pem
# Extract public key from certificate
openssl x509 -in cert.pem -pubkey -noout -out cert-pub.pem
# Compare (should be identical)
diff key-pub.pem cert-pub.pem- Verify XML wasn't modified:
# Inspect canonical hash
jux-inspect -i junit.xml
# Sign again and compare hashes
jux-sign -i original.xml -o signed2.xml --key key.pem
jux-inspect -i signed2.xml- Check file permissions:
ls -la ~/.ssh/jux/
# Private key should be 600, certificate should be 644✓ Checkpoint: Debugging strategies understood
cat > Makefile << 'EOF'
.PHONY: test test-dev test-staging test-prod verify-staging verify-prod clean
# Development testing (no caching)
test-dev:
pytest \
--jux-config config/development.toml \
--jux-key ~/.ssh/jux/dev-key.pem \
--jux-cert ~/.ssh/jux/dev-cert.pem
# Staging testing (with caching)
test-staging:
pytest \
--jux-config config/staging.toml \
--jux-key ~/.ssh/jux/staging-key.pem \
--jux-cert ~/.ssh/jux/staging-cert.pem
# Production testing (with caching)
test-prod:
pytest \
--jux-config config/production.toml \
--jux-key ~/.ssh/jux/prod-key.pem \
--jux-cert ~/.ssh/jux/prod-cert.pem
# Verify staging report
verify-staging:
jux-verify \
-i test-results/junit.xml \
--cert ~/.ssh/jux/staging-cert.pem
# Verify production report
verify-prod:
jux-verify \
-i test-results/junit.xml \
--cert ~/.ssh/jux/prod-cert.pem
# Clean up
clean:
rm -rf test-results/
rm -rf .pytest_cache/
rm -rf htmlcov/
rm -rf .coverage
jux-cache purge
EOFUsage:
# Run staging tests
make test-staging
# Verify staging report
make verify-staging
# Clean up
make clean# Add to ~/.bashrc or ~/.zshrc
cat >> ~/.bashrc << 'EOF'
# pytest-jux aliases
alias pytest-dev='pytest --jux-config config/development.toml'
alias pytest-staging='pytest --jux-config config/staging.toml'
alias pytest-prod='pytest --jux-config config/production.toml'
alias jux-verify-staging='jux-verify --cert ~/.ssh/jux/staging-cert.pem'
alias jux-verify-prod='jux-verify --cert ~/.ssh/jux/prod-cert.pem'
EOF
# Reload shell
source ~/.bashrc✓ Checkpoint: Advanced configuration completed
Best Practice: Use different keys for each environment
# Development: Short-lived, low security
jux-keygen --type rsa --bits 2048 --days-valid 90
# Staging: Medium security
jux-keygen --type rsa --bits 4096 --days-valid 180
# Production: High security
jux-keygen --type rsa --bits 4096 --days-valid 365| Environment | Storage Mode | Rationale |
|---|---|---|
| Development | disabled |
Frequent test runs, reports ephemeral |
| Staging | auto |
Cache for debugging, test report consistency |
| Production | auto |
Archive for compliance, auditing |
| CI/CD | disabled |
Reports uploaded as artifacts |
# Rotate keys quarterly
# 1. Generate new keys
jux-keygen --output new-prod-key.pem --cert
# 2. Update configuration
# Edit config/production.toml with new key paths
# 3. Update CI/CD secrets
# Update GitHub Secrets with new key
# 4. Archive old key (for verifying historical reports)
mv ~/.ssh/jux/prod-key.pem ~/.ssh/jux/prod-key-2025Q3.pem.archived# Add verification step to CI/CD
- name: Verify all signed reports
run: |
for report in test-results/*.xml; do
echo "Verifying $report"
jux-verify -i "$report" --cert ~/.ssh/jux/ci-key.crt
done✓ Checkpoint: Best practices understood
✅ Completed:
- Integrated pytest-jux into a realistic Python project
- Configured multi-environment signing (dev, staging, production)
- Set up GitHub Actions CI/CD with automated signing
- Implemented secure key management strategies
- Configured report storage and archival
- Learned debugging techniques for integration issues
- Created advanced configuration with Makefile and aliases
- Applied integration testing best practices
🎓 Skills Learned:
- Multi-environment configuration
- CI/CD integration (GitHub Actions)
- Secure key management (ephemeral vs persistent)
- Report storage strategies
- Debugging integration issues
- Advanced pytest-jux configuration
Now that you've mastered integration, try:
- Custom Signing Workflows: Implement custom signing logic
- Multi-Environment Configuration Guide: Deep dive into configuration
- CI/CD Deployment Guide: More CI/CD examples (GitLab CI, Jenkins)
- Security Best Practices: Production security guidelines
- Key Management Guide: Enterprise key management
- Monitoring Guide: Monitor signing operations
Problem: pytest-jux not installed in CI
Solution: Ensure installation step:
- name: Install dependencies
run: pip install -e ".[dev]"Problem: Private key has wrong permissions (too open)
Solution: Fix permissions after creating key:
- name: Generate ephemeral signing key
run: |
jux-keygen --output ~/.ssh/jux/ci-key.pem --cert
chmod 600 ~/.ssh/jux/ci-key.pemProblem: Config file path incorrect
Solution: Use absolute path or check working directory:
# Absolute path
pytest --jux-config "$(pwd)/config/development.toml"
# Or set environment variable
export JUX_CONFIG_FILE="$(pwd)/config/development.toml"
pytest# Remove project directory
cd ..
rm -rf pytest-jux-integration-demo
# Remove ephemeral keys (keep prod keys if needed)
rm ~/.ssh/jux/dev-key.pem ~/.ssh/jux/dev-cert.pem
# Clear cache
jux-cache purgeWas this tutorial helpful? Have suggestions?
- Report Issues: https://github.com/jux-tools/pytest-jux/issues
- Contribute: PRs welcome!
Tutorial Version: 1.0 pytest-jux Version: 0.1.9 Last Updated: 2025-10-20