Skip to content

Commit 68c48ed

Browse files
tlbauer2tlbauer2
authored
Merge pull request #15 from garland3/feature/ci-cd-pipeline
Add comprehensive CI/CD pipeline with multi-platform support
2 parents 683beef + 25bc41e commit 68c48ed

9 files changed

Lines changed: 1123 additions & 27 deletions

File tree

‎.bandit‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
[bandit]
2+
# Bandit configuration file
3+
exclude_dirs = [
4+
"tests",
5+
"build",
6+
".git"
7+
]
8+
9+
# Skip these test IDs
10+
skips = [
11+
# B101: Test for use of assert
12+
"B101"
13+
]
14+
15+
# Confidence levels: LOW, MEDIUM, HIGH
16+
confidence = "HIGH"
17+
18+
# Severity levels: LOW, MEDIUM, HIGH
19+
severity = "MEDIUM"

‎.dockerignore‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Version control
2+
.git
3+
.gitignore
4+
5+
# Documentation
6+
*.md
7+
docs/
8+
9+
# Development tools
10+
.vscode/
11+
.idea/
12+
13+
# Python
14+
__pycache__/
15+
*.py[cod]
16+
*$py.class
17+
*.so
18+
.Python
19+
build/
20+
develop-eggs/
21+
dist/
22+
downloads/
23+
eggs/
24+
.eggs/
25+
lib/
26+
lib64/
27+
parts/
28+
sdist/
29+
var/
30+
wheels/
31+
share/python-wheels/
32+
*.egg-info/
33+
.installed.cfg
34+
*.egg
35+
MANIFEST
36+
37+
# Testing
38+
.tox/
39+
.coverage
40+
.coverage.*
41+
.cache
42+
.pytest_cache/
43+
coverage.xml
44+
*.cover
45+
htmlcov/
46+
47+
# Virtual environments
48+
.env
49+
.venv
50+
env/
51+
venv/
52+
ENV/
53+
env.bak/
54+
venv.bak/
55+
56+
# IDE
57+
.vscode/
58+
.idea/
59+
*.swp
60+
*.swo
61+
*~
62+
63+
# OS
64+
.DS_Store
65+
.DS_Store?
66+
._*
67+
.Spotlight-V100
68+
.Trashes
69+
ehthumbs.db
70+
Thumbs.db
71+
72+
# Logs
73+
*.log
74+
75+
# Security
76+
.bandit
77+
bandit-report.json
78+
safety-report.json
79+
trivy-results.sarif
80+
81+
# CI/CD
82+
.github/

‎.github/README.md‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# GitHub CI/CD Configuration
2+
3+
This directory contains the GitHub Actions workflows and configuration for the TalkPipe project's automated CI/CD pipeline.
4+
5+
## Files Overview
6+
7+
### Workflows (`workflows/`)
8+
9+
#### `ci-cd.yml` - Main CI/CD Pipeline
10+
Comprehensive pipeline that runs on:
11+
- Pushes to `main` and `develop` branches
12+
- Pull requests to `main` and `develop`
13+
- GitHub releases
14+
15+
**Pipeline Jobs:**
16+
1. **Test** - Multi-version Python testing (3.11, 3.12) with coverage
17+
2. **Security Scan** - SAST and dependency vulnerability scanning
18+
3. **Build Container** - Docker image build/push with container security scan
19+
4. **CodeQL Analysis** - GitHub's semantic code analysis
20+
5. **Publish Package** - Automated PyPI publishing on releases
21+
22+
### Configuration Files
23+
24+
- **`dependabot.yml`** - Automated dependency updates (weekly schedule)
25+
- **`SECURITY.md`** - Security policy and vulnerability reporting guidelines
26+
27+
### Root Level Security Files
28+
29+
- **`.bandit`** - Configuration for Bandit static security analysis
30+
- **`.dockerignore`** - Optimized Docker build context exclusions
31+
32+
## Security Scanning
33+
34+
The pipeline includes multiple layers of security scanning:
35+
36+
- **Bandit** - Python static analysis security testing (SAST)
37+
- **Safety** - Python dependency vulnerability scanning
38+
- **Trivy** - Container image vulnerability scanning
39+
- **CodeQL** - Semantic code analysis for security issues
40+
- **Dependabot** - Automated dependency update PRs
41+
42+
## Container Registry
43+
44+
Docker images are built and pushed to GitHub Container Registry:
45+
```
46+
ghcr.io/sandialabs/talkpipe:latest
47+
ghcr.io/sandialabs/talkpipe:<branch-name>
48+
ghcr.io/sandialabs/talkpipe:<version>
49+
```
50+
51+
## Required Repository Secrets
52+
53+
To enable full functionality, set these secrets in your GitHub repository settings (`Settings > Secrets and variables > Actions`):
54+
55+
### Required for PyPI Publishing
56+
- **`PYPI_API_TOKEN`** - PyPI API token for automated package publishing
57+
- Create at: https://pypi.org/manage/account/token/
58+
- Scope: Entire account or specific to talkpipe project
59+
- Used in: Package publishing job (triggered on releases)
60+
61+
### Automatic Secrets (No Action Required)
62+
- **`GITHUB_TOKEN`** - Automatically provided by GitHub Actions
63+
- Used for: Container registry authentication, uploading artifacts, CodeQL results
64+
65+
## Setup Instructions
66+
67+
1. **Enable GitHub Container Registry** (if not already enabled):
68+
- Go to repository `Settings > General`
69+
- Scroll to "Features" section
70+
- Ensure "Packages" is enabled
71+
72+
2. **Set PyPI Token**:
73+
```bash
74+
# In repository Settings > Secrets and variables > Actions
75+
# Add new repository secret:
76+
Name: PYPI_API_TOKEN
77+
Secret: pypi-your-token-here
78+
```
79+
80+
3. **Configure Dependabot** (optional customization):
81+
- Edit `.github/dependabot.yml` to adjust reviewers/assignees
82+
- Default: weekly updates on Mondays at 9 AM UTC
83+
84+
## Testing Locally
85+
86+
Before pushing, you can test components locally:
87+
88+
```bash
89+
# Run tests with coverage (matches CI)
90+
pytest --cov=src --cov-report=xml --cov-report=html
91+
92+
# Run security scans
93+
bandit -r src/
94+
safety check
95+
96+
# Build container (matches CI)
97+
docker build -t talkpipe:local .
98+
99+
# Run container security scan
100+
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
101+
aquasec/trivy image talkpipe:local
102+
```
103+
104+
## Workflow Triggers
105+
106+
| Event | Trigger | Jobs Run |
107+
|-------|---------|----------|
108+
| Push to main/develop | Automatic | All jobs |
109+
| Pull Request | Automatic | All except publish |
110+
| Release published | Automatic | All jobs + PyPI publish |
111+
| Manual trigger | `workflow_dispatch` | All jobs |
112+
113+
## Monitoring
114+
115+
- **Test Results**: Visible in Actions tab and PR checks
116+
- **Coverage Reports**: Uploaded to Codecov (if configured)
117+
- **Security Issues**: Reported in Security tab (CodeQL, Trivy)
118+
- **Container Images**: Available in Packages tab
119+
120+
## Troubleshooting
121+
122+
**Common Issues:**
123+
124+
1. **PyPI Publishing Fails**:
125+
- Verify `PYPI_API_TOKEN` is set correctly
126+
- Ensure token has sufficient permissions
127+
- Check package version doesn't already exist
128+
129+
2. **Container Build Fails**:
130+
- Check Dockerfile syntax
131+
- Verify base image availability
132+
- Review build logs in Actions tab
133+
134+
3. **Tests Fail**:
135+
- Run tests locally first
136+
- Check dependency compatibility
137+
- Review test logs in Actions tab
138+
139+
4. **Security Scans Fail**:
140+
- Review Bandit/Safety reports
141+
- Update vulnerable dependencies
142+
- Add exclusions to `.bandit` if needed
143+
144+
For additional help, check the Actions tab logs or create an issue in the repository.

‎.github/SECURITY.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Security Policy
2+
3+
## Supported Versions
4+
5+
We actively support security updates for the following versions:
6+
7+
| Version | Supported |
8+
| ------- | ------------------ |
9+
| Latest | :white_check_mark: |
10+
11+
## Reporting a Vulnerability
12+
13+
If you discover a security vulnerability in TalkPipe, please report it responsibly:
14+
15+
### Private Disclosure
16+
17+
1. **DO NOT** create a public GitHub issue for security vulnerabilities
18+
2. Email security reports to: tlbauer@sandia.gov
19+
3. Include the following information:
20+
- Description of the vulnerability
21+
- Steps to reproduce the issue
22+
- Potential impact
23+
- Suggested fix (if available)
24+
25+
### What to Expect
26+
27+
- **Response Time**: We aim to acknowledge receipt within 2 business days
28+
- **Initial Assessment**: Initial security assessment within 5 business days
29+
- **Updates**: We will provide updates on our progress every 7 days until resolution
30+
- **Disclosure**: After the vulnerability is fixed, we will coordinate public disclosure
31+
32+
### Security Best Practices
33+
34+
When using TalkPipe:
35+
36+
1. **Keep Dependencies Updated**: Regularly update TalkPipe and its dependencies
37+
2. **Secure Configuration**: Follow security best practices for API keys and configuration
38+
3. **Container Security**: When using Docker, ensure base images are up to date
39+
4. **Network Security**: Implement appropriate network security measures
40+
5. **Access Control**: Limit access to TalkPipe deployments to authorized users only
41+
42+
### Security Features
43+
44+
TalkPipe includes several security measures:
45+
46+
- Dependency scanning via Safety and Bandit
47+
- Container vulnerability scanning with Trivy
48+
- Static code analysis with CodeQL
49+
- Automated dependency updates via Dependabot
50+
- Non-root container execution
51+
- Input validation and sanitization
52+
53+
### Acknowledgments
54+
55+
We appreciate security researchers who responsibly disclose vulnerabilities. Contributors will be acknowledged (unless they prefer to remain anonymous) in our security advisories and release notes.

‎.github/dependabot.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
version: 2
2+
updates:
3+
# Enable version updates for Python dependencies
4+
- package-ecosystem: "pip"
5+
directory: "/"
6+
schedule:
7+
interval: "weekly"
8+
day: "monday"
9+
time: "09:00"
10+
reviewers:
11+
- "tlbauer"
12+
assignees:
13+
- "tlbauer"
14+
commit-message:
15+
prefix: "deps"
16+
include: "scope"
17+
18+
# Enable version updates for GitHub Actions
19+
- package-ecosystem: "github-actions"
20+
directory: "/"
21+
schedule:
22+
interval: "weekly"
23+
day: "monday"
24+
time: "09:00"
25+
reviewers:
26+
- "tlbauer"
27+
assignees:
28+
- "tlbauer"
29+
commit-message:
30+
prefix: "ci"
31+
include: "scope"
32+
33+
# Enable version updates for Docker
34+
- package-ecosystem: "docker"
35+
directory: "/"
36+
schedule:
37+
interval: "weekly"
38+
day: "monday"
39+
time: "09:00"
40+
reviewers:
41+
- "tlbauer"
42+
assignees:
43+
- "tlbauer"
44+
commit-message:
45+
prefix: "docker"
46+
include: "scope"

0 commit comments

Comments
 (0)