Modernize Release Process: Complete Automation & CI/CD Pipeline

[aaronlippold]

Modernize Vulcan Release Process: Complete Automation & CI/CD Pipeline

📋 Overview

This issue proposes a complete redesign of Vulcan's release process to eliminate manual steps, improve reliability, and implement modern CI/CD best practices. The current process requires significant manual intervention and is prone to human error.

🔍 Current State Analysis

Existing Release Workflow

Based on Wiki: Releasing Vulcan:

1. Automated Draft Creation: Every 14 days via GitHub Actions
2. Manual Review: Developer reviews/edits draft release
3. Version Management: Update multiple files manually
4. Testing: Automated test suite + manual staging verification
5. Publishing: Manual release publication

Current Automation (✅ Working)

• Draft Release Creation (.github/workflows/create-draft-release.yml)

  • Runs every 14 days automatically
  • Auto-calculates next patch version (v2.1.8 → v2.1.9)
  • Downloads source artifacts
  • Creates GitHub draft release

• Docker Publishing (.github/workflows/push-to-docker.yml)

  • Triggers on successful test suite OR published release
  • Publishes mitre/vulcan:latest on master pushes
  • Publishes mitre/vulcan:v2.x.x on release publication

• Test Automation (.github/workflows/run-tests.yml)

  • Runs on PRs, pushes, draft releases
  • PostgreSQL + LDAP container testing
  • RSpec + RuboCop + Yarn lint

Current Pain Points (🔴 Issues)

1. Multiple Manual File Updates: Developer must update 4+ files for each release

   • VERSION file
   • package.json version
   • README.md latest release section
   • CHANGELOG.md entries

2. Version Sync Problems: Files easily get out of sync

3. Manual Changelog: No automated changelog generation

4. Semantic Versioning Gaps: Only auto-increments patch versions

5. Release Notes Manual: Completely manual process

6. No Pre-release Testing: No automated staging deployment validation

7. Fixed Schedule: 14-day schedule doesn't align with development pace

🎯 Proposed Solution: Complete Release Process Redesign

Design Principles

1. Zero Manual File Updates - All version bumps automated
2. Conventional Commits Driven - Release decisions based on commit analysis
3. Multi-Environment Validation - Staging → Production pipeline
4. Rollback Ready - Quick recovery from failed releases
5. Container-First - Modern deployment patterns
6. Security-First - Automated vulnerability scanning

Architecture Overview

graph TD
    A[Developer Commit] --> B[Commit Analysis]
    B --> C{Should Release?}
    C -->|Yes| D[Version Management]
    C -->|No| E[Wait for more commits]
    D --> F[Build & Test]
    F --> G[Security Scanning]
    G --> H[Build Artifacts]
    H --> I[Deploy Staging]
    I --> J[Staging Validation]
    J --> K[Manual Approval Gate]
    K --> L[Create GitHub Release]
    L --> M[Deploy Production]
    M --> N[Post-deployment Validation]
    N --> O[Notify Stakeholders]
    
    P[Emergency Rollback] -.-> M

Loading

🚀 Implementation Plan

Phase 1: Foundation (Week 1-2)

• Implement conventional commit standards
• Create automated version management workflow
• Add semantic-release integration
• Automated changelog generation

Phase 2: Enhanced Validation (Week 3-4)

• Multi-stage validation pipeline
• Automated security scanning
• Container vulnerability assessment
• Build artifact validation

Phase 3: Deployment Pipeline (Week 5-6)

• Staging deployment automation
• Integration test validation
• Manual approval gates
• Production deployment automation

Phase 4: Monitoring & Recovery (Week 7-8)

• Post-deployment validation
• Stakeholder notifications
• Emergency rollback system
• Incident management integration

📝 Detailed Implementation

1. Conventional Commit Standards

Implement standardized commit message format:

feat: add OIDC auto-discovery (#672)           # → Minor version bump
fix: resolve LDAP authentication issue (#669)  # → Patch version bump  
feat!: migrate to rails-settings-cached       # → Major version bump
docs: update installation guide               # → No version bump
ci: update GitHub Actions versions            # → No version bump

2. Automated Version Management

Create workflow to atomically update all version files:

# .github/workflows/version-management.yml
name: Automated Version Management

jobs:
  version_bump:
    runs-on: ubuntu-latest
    steps:
      - name: Update all version files atomically
        run: |
          NEW_VERSION="${{ steps.version.outputs.new_version }}"
          
          # Update VERSION file
          echo "$NEW_VERSION" > VERSION
          
          # Update package.json
          jq ".version = \"${NEW_VERSION#v}\"" package.json > package.json.tmp
          mv package.json.tmp package.json
          
          # Update README.md latest release section
          sed -i "s/Latest Release: \[v[0-9]\+\.[0-9]\+\.[0-9]\+\]/Latest Release: [$NEW_VERSION]/" README.md
          
          # Update documentation
          find docs/ -name "*.yaml" -o -name "*.yml" | xargs sed -i "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:$NEW_VERSION/g"

3. Multi-Stage Validation Pipeline

# .github/workflows/release-validation.yml
name: Release Validation Pipeline

jobs:
  # Stage 1: Build & Test
  build_and_test:
    strategy:
      matrix:
        ruby: ['3.0', '3.1', '3.2']
        node: ['16', '18', '20']
    steps:
      - name: Run comprehensive test suite
      - name: Security scanning
      - name: Vulnerability assessment
        
  # Stage 2: Build Release Artifacts  
  build_artifacts:
    steps:
      - name: Build multi-platform Docker images
      - name: Generate SBOM
      
  # Stage 3: Staging Deployment
  deploy_staging:
    environment: staging
    steps:
      - name: Deploy to staging
      - name: Run integration tests
      - name: Performance validation
      - name: Security validation
      
  # Stage 4: Manual Approval Gate
  approval_gate:
    environment: production-approval
    steps:
      - name: Request manual approval from team leads

4. Emergency Rollback System

# .github/workflows/emergency-rollback.yml
name: Emergency Rollback

on:
  workflow_dispatch:
    inputs:
      target_version:
        description: 'Version to rollback to'
        required: true
      reason:
        description: 'Rollback reason'
        required: true

jobs:
  rollback:
    environment: production-emergency
    steps:
      - name: Validate rollback target
      - name: Execute rollback
      - name: Validate rollback success
      - name: Create incident report

📊 Expected Benefits

Automation Benefits

• Zero Manual Steps: Complete automation from commit → production
• Consistent Quality: Every release follows same validation pipeline
• Fast Recovery: Automated rollback in <5 minutes
• Security First: Automated vulnerability scanning & validation

Developer Experience

• Simple: git commit -m "feat: new feature" → automatic release
• Predictable: Clear release timeline and process
• Safe: Staging validation + approval gates
• Transparent: Full audit trail and notifications

Enterprise Ready

• Compliance: Full audit trail and approval workflows
• Security: Automated scanning and validation
• Reliability: Multi-stage validation and rollback capabilities
• Monitoring: Comprehensive notifications and metrics

Cost Savings

• Time: 3 hours → 15 minutes per release
• Risk: Eliminate human error in version management
• Quality: Consistent testing and validation
• Recovery: Fast rollback vs lengthy incident response

🔧 Required Tools & Dependencies

• semantic-release: Automated version management + changelog
• conventional-changelog: Generate changelogs from conventional commits
• GitHub CLI: Streamline GitHub operations
• Docker Buildx: Multi-platform container builds
• Trivy: Container vulnerability scanning
• OWASP ZAP: Security validation

📅 Timeline

• Target Start: After v2.2.0 settings migration release
• Estimated Duration: 8 weeks
• Target Completion: Before v2.3.0 release

🎯 Success Criteria

• Zero manual file updates required for releases
• Automated semantic versioning based on conventional commits
• Automated changelog generation
• Multi-stage validation pipeline operational
• Emergency rollback system tested and documented
• Release time reduced from 3 hours to <15 minutes
• All team members trained on new process

📚 References

• Current Release Wiki
• Conventional Commits
• Semantic Release
• GitHub Actions Best Practices

🏷️ Related Issues

• This modernization enables the container-first migration tooling planned in the settings migration work
• Supports the enterprise configuration management roadmap
• Aligns with Vulcan modernization initiatives

---

Priority: Medium (after v2.2.0 settings migration)
Effort: Large (8 weeks)
Impact: High (major developer experience improvement)

[aaronlippold]

📁 Complete Implementation Files & Scripts

This comment provides the detailed file inventory and helper scripts for the modernized release process.

🗂️ New Files to Create

GitHub Workflows

.github/workflows/
├── commit-analysis.yml           # Analyze commits for release planning
├── release-orchestrator.yml      # Main release decision engine
├── version-management.yml        # Automated version file updates
├── release-validation.yml        # Multi-stage validation pipeline
├── production-release.yml        # GitHub release creation & deployment
├── emergency-rollback.yml        # Rollback system
└── staging-cleanup.yml          # Clean up old staging deployments

Configuration Files

.releaserc.json                   # Semantic-release configuration
.commitlintrc.json               # Commit message validation
.github/release.yml              # GitHub release template
.github/CODEOWNERS              # Require approval for release files

Helper Scripts

scripts/
├── update-version-files.sh      # Atomic version updates
├── release-helper.sh           # Interactive release management
├── rollback-helper.sh          # Emergency rollback assistant
├── validate-release.sh         # Pre-release validation
├── generate-release-notes.sh   # Enhanced release notes
└── setup-release-tools.sh      # One-time setup for release tools

📝 Complete File Contents

1. Semantic Release Configuration

// .releaserc.json
{
  "branches": ["master"],
  "repositoryUrl": "https://github.com/mitre/vulcan",
  "plugins": [
    [
      "@semantic-release/commit-analyzer",
      {
        "preset": "conventionalcommits",
        "releaseRules": [
          { "type": "feat", "release": "minor" },
          { "type": "fix", "release": "patch" },
          { "type": "perf", "release": "patch" },
          { "type": "revert", "release": "patch" },
          { "type": "docs", "release": false },
          { "type": "style", "release": false },
          { "type": "chore", "release": false },
          { "type": "refactor", "release": "patch" },
          { "type": "test", "release": false },
          { "type": "build", "release": false },
          { "type": "ci", "release": false }
        ]
      }
    ],
    [
      "@semantic-release/release-notes-generator",
      {
        "preset": "conventionalcommits",
        "presetConfig": {
          "types": [
            { "type": "feat", "section": "🚀 Features" },
            { "type": "fix", "section": "🐛 Bug Fixes" },
            { "type": "perf", "section": "⚡ Performance" },
            { "type": "revert", "section": "⏪ Reverts" },
            { "type": "docs", "section": "📚 Documentation" },
            { "type": "style", "section": "💄 Styles" },
            { "type": "refactor", "section": "♻️ Code Refactoring" },
            { "type": "test", "section": "✅ Tests" },
            { "type": "build", "section": "🔨 Build System" },
            { "type": "ci", "section": "🔧 CI/CD" },
            { "type": "chore", "section": "🏗️ Chores" }
          ]
        }
      }
    ],
    [
      "@semantic-release/changelog",
      {
        "changelogFile": "CHANGELOG.md"
      }
    ],
    [
      "@semantic-release/exec",
      {
        "prepareCmd": "scripts/update-version-files.sh ${nextRelease.version}",
        "publishCmd": "scripts/post-release-tasks.sh ${nextRelease.version}"
      }
    ],
    [
      "@semantic-release/github",
      {
        "assets": [
          {
            "path": "CHANGELOG.md",
            "label": "Changelog"
          }
        ]
      }
    ],
    [
      "@semantic-release/git",
      {
        "assets": [
          "VERSION",
          "package.json",
          "CHANGELOG.md",
          "README.md",
          "docs/**/*.yaml",
          "docs/**/*.yml"
        ],
        "message": "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
      }
    ]
  ]
}

2. Version Update Script

#!/bin/bash
# scripts/update-version-files.sh
# Atomically update all version files

set -euo pipefail

NEW_VERSION=$1
if [[ -z "$NEW_VERSION" ]]; then
    echo "❌ Error: Version parameter required"
    echo "Usage: $0 <version>"
    exit 1
fi

# Remove 'v' prefix if present
NEW_VERSION=${NEW_VERSION#v}

echo "🔄 Updating all version files to v$NEW_VERSION"

# 1. Update VERSION file
echo "v$NEW_VERSION" > VERSION
echo "✅ Updated VERSION file"

# 2. Update package.json
if [[ -f "package.json" ]]; then
    jq ".version = \"$NEW_VERSION\"" package.json > package.json.tmp
    mv package.json.tmp package.json
    echo "✅ Updated package.json"
fi

# 3. Update README.md latest release section
if [[ -f "README.md" ]]; then
    sed -i.bak "s/Latest Release: \[v[0-9]\+\.[0-9]\+\.[0-9]\+\]/Latest Release: [v$NEW_VERSION]/" README.md
    sed -i.bak "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:v$NEW_VERSION/" README.md
    rm README.md.bak 2>/dev/null || true
    echo "✅ Updated README.md"
fi

# 4. Update documentation files
if [[ -d "docs" ]]; then
    find docs/ -name "*.yaml" -o -name "*.yml" | while read -r file; do
        sed -i.bak "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:v$NEW_VERSION/g" "$file"
        rm "${file}.bak" 2>/dev/null || true
    done
    echo "✅ Updated documentation files"
fi

# 5. Update Helm charts if they exist
if [[ -d "charts" ]]; then
    find charts/ -name "Chart.yaml" | while read -r chart; do
        sed -i.bak "s/version: [0-9]\+\.[0-9]\+\.[0-9]\+/version: $NEW_VERSION/" "$chart"
        sed -i.bak "s/appVersion: [0-9]\+\.[0-9]\+\.[0-9]\+/appVersion: $NEW_VERSION/" "$chart"
        rm "${chart}.bak" 2>/dev/null || true
    done
    echo "✅ Updated Helm charts"
fi

echo "🎉 All version files updated successfully to v$NEW_VERSION"
echo ""
echo "📋 Updated files:"
echo "  - VERSION"
echo "  - package.json"
echo "  - README.md"
echo "  - docs/**/*.{yaml,yml}"
[[ -d "charts" ]] && echo "  - charts/**/Chart.yaml"

3. Interactive Release Helper Script

#!/bin/bash
# scripts/release-helper.sh
# Interactive helper for manual release management

set -euo pipefail

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

echo -e "${BLUE}🚀 Vulcan Release Helper${NC}"
echo "=================================="

# Check if we're on master branch
current_branch=$(git branch --show-current)
if [[ "$current_branch" != "master" ]]; then
    echo -e "${RED}❌ Error: Must be on master branch${NC}"
    echo "Current branch: $current_branch"
    exit 1
fi

# Check for uncommitted changes
if [[ -n $(git status --porcelain) ]]; then
    echo -e "${RED}❌ Error: Uncommitted changes detected${NC}"
    echo "Please commit or stash your changes first"
    git status --short
    exit 1
fi

# Get current version
current_version=$(cat VERSION)
echo -e "📋 Current version: ${GREEN}$current_version${NC}"

# Analyze commits since last release
echo ""
echo "📊 Analyzing commits since last release..."
commits=$(git log --oneline $(git describe --tags --abbrev=0)..HEAD)
if [[ -z "$commits" ]]; then
    echo -e "${YELLOW}⚠️  No commits since last release${NC}"
    exit 0
fi

echo "Recent commits:"
echo "$commits" | head -10

# Count commit types
feat_count=$(echo "$commits" | grep -c "^[a-f0-9]* feat" || true)
fix_count=$(echo "$commits" | grep -c "^[a-f0-9]* fix" || true)
breaking_count=$(echo "$commits" | grep -c "!" || true)

echo ""
echo "📈 Commit analysis:"
echo "  - Features: $feat_count"
echo "  - Bug fixes: $fix_count"
echo "  - Breaking changes: $breaking_count"

# Suggest version bump
if [[ $breaking_count -gt 0 ]]; then
    suggested="major"
elif [[ $feat_count -gt 0 ]]; then
    suggested="minor"
elif [[ $fix_count -gt 0 ]]; then
    suggested="patch"
else
    suggested="patch"
fi

echo -e "💡 Suggested release type: ${GREEN}$suggested${NC}"

# Interactive release type selection
echo ""
echo "🎯 Select release type:"
echo "1) patch (bug fixes)"
echo "2) minor (new features)"
echo "3) major (breaking changes)"
echo "4) auto (use suggestion: $suggested)"
echo "5) custom version"
echo "6) cancel"

read -p "Enter choice [1-6]: " choice

case $choice in
    1) release_type="patch" ;;
    2) release_type="minor" ;;
    3) release_type="major" ;;
    4) release_type="$suggested" ;;
    5) 
        read -p "Enter custom version (e.g., 2.3.0): " custom_version
        release_type="custom"
        ;;
    6) 
        echo "❌ Release cancelled"
        exit 0
        ;;
    *)
        echo "❌ Invalid choice"
        exit 1
        ;;
esac

# Calculate new version
if [[ "$release_type" == "custom" ]]; then
    new_version="v$custom_version"
else
    # Parse current version
    current_clean=${current_version#v}
    IFS='.' read -ra VERSION_PARTS <<< "$current_clean"
    major=${VERSION_PARTS[0]}
    minor=${VERSION_PARTS[1]}
    patch=${VERSION_PARTS[2]}

    case $release_type in
        "major") major=$((major + 1)); minor=0; patch=0 ;;
        "minor") minor=$((minor + 1)); patch=0 ;;
        "patch") patch=$((patch + 1)) ;;
    esac

    new_version="v${major}.${minor}.${patch}"
fi

echo ""
echo -e "📦 Preparing release: ${GREEN}$current_version → $new_version${NC}"

# Confirmation
read -p "🤔 Proceed with release? [y/N]: " confirm
if [[ ! "$confirm" =~ ^[Yy]$ ]]; then
    echo "❌ Release cancelled"
    exit 0
fi

# Trigger GitHub workflow
echo ""
echo "🚀 Triggering release workflow..."
gh workflow run release-orchestrator.yml \
    -f release_type="$release_type" \
    -f target_version="$new_version"

echo -e "${GREEN}✅ Release workflow triggered!${NC}"
echo ""
echo "📋 Next steps:"
echo "  1. Monitor workflow: gh run list --workflow=release-orchestrator.yml"
echo "  2. Check staging deployment when ready"
echo "  3. Approve production release when validation passes"
echo ""
echo "🔗 Useful commands:"
echo "  gh run list --workflow=release-orchestrator.yml"
echo "  gh run view --log"
echo "  gh pr list --state=open"

4. Emergency Rollback Helper

#!/bin/bash
# scripts/rollback-helper.sh
# Emergency rollback assistant

set -euo pipefail

RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

echo -e "${RED}🚨 Emergency Rollback Helper${NC}"
echo "====================================="

# List recent releases
echo "📋 Recent releases:"
gh release list --limit 10

echo ""
read -p "Enter version to rollback to (e.g., v2.1.8): " target_version

if [[ -z "$target_version" ]]; then
    echo -e "${RED}❌ Version required${NC}"
    exit 1
fi

# Validate version exists
if ! gh release view "$target_version" >/dev/null 2>&1; then
    echo -e "${RED}❌ Version $target_version not found${NC}"
    exit 1
fi

echo ""
read -p "Rollback reason: " reason

if [[ -z "$reason" ]]; then
    echo -e "${RED}❌ Reason required for audit trail${NC}"
    exit 1
fi

echo ""
echo -e "${YELLOW}⚠️  WARNING: This will rollback production to $target_version${NC}"
echo "Reason: $reason"
echo ""
read -p "Are you absolutely sure? Type 'ROLLBACK' to confirm: " confirm

if [[ "$confirm" != "ROLLBACK" ]]; then
    echo "❌ Rollback cancelled"
    exit 0
fi

# Trigger rollback workflow
echo ""
echo "🚨 Executing emergency rollback..."
gh workflow run emergency-rollback.yml \
    -f target_version="$target_version" \
    -f reason="$reason"

echo -e "${GREEN}✅ Rollback initiated!${NC}"
echo ""
echo "📋 Monitor rollback progress:"
echo "  gh run list --workflow=emergency-rollback.yml"
echo "  gh run view --log"

5. Pre-release Validation Script

#!/bin/bash
# scripts/validate-release.sh
# Comprehensive pre-release validation

set -euo pipefail

echo "🔍 Pre-Release Validation"
echo "========================="

exit_code=0

# Check 1: Git status
echo "📋 Checking git status..."
if [[ -n $(git status --porcelain) ]]; then
    echo "❌ Uncommitted changes detected"
    exit_code=1
else
    echo "✅ Git working tree clean"
fi

# Check 2: Tests
echo ""
echo "🧪 Running test suite..."
if bundle exec rspec --format progress; then
    echo "✅ All tests pass"
else
    echo "❌ Test failures detected"
    exit_code=1
fi

# Check 3: Linting
echo ""
echo "📝 Running linters..."
if bundle exec rubocop; then
    echo "✅ RuboCop passed"
else
    echo "❌ RuboCop violations"
    exit_code=1
fi

if yarn lint; then
    echo "✅ JavaScript lint passed"
else
    echo "❌ JavaScript lint violations"
    exit_code=1
fi

# Check 4: Security
echo ""
echo "🔒 Security scan..."
if bundle audit check --update; then
    echo "✅ No known vulnerabilities"
else
    echo "❌ Security vulnerabilities detected"
    exit_code=1
fi

# Check 5: Docker build
echo ""
echo "🐳 Testing Docker build..."
if docker build -t vulcan-test .; then
    echo "✅ Docker build successful"
    docker rmi vulcan-test
else
    echo "❌ Docker build failed"
    exit_code=1
fi

# Check 6: Version consistency
echo ""
echo "📦 Checking version consistency..."
version_file=$(cat VERSION)
package_version="v$(jq -r '.version' package.json)"

if [[ "$version_file" == "$package_version" ]]; then
    echo "✅ Version files consistent: $version_file"
else
    echo "❌ Version mismatch: VERSION=$version_file, package.json=$package_version"
    exit_code=1
fi

echo ""
if [[ $exit_code -eq 0 ]]; then
    echo "🎉 All validations passed - ready for release!"
else
    echo "❌ Validation failures detected - fix before releasing"
fi

exit $exit_code

🎯 Helper Script for Current Release Process

Since you asked about a helper script for the current release process, here's one we could create right now:

#!/bin/bash
# scripts/current-release-helper.sh
# Helper for current manual release process

set -euo pipefail

GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
NC='\033[0m'

echo -e "${BLUE}📦 Current Vulcan Release Helper${NC}"
echo "===================================="

# Get current version
current_version=$(cat VERSION)
echo -e "Current version: ${GREEN}$current_version${NC}"

# Parse version and calculate next
current_clean=${current_version#v}
IFS='.' read -ra VERSION_PARTS <<< "$current_clean"
major=${VERSION_PARTS[0]}
minor=${VERSION_PARTS[1]}
patch=${VERSION_PARTS[2]}

next_patch="v${major}.${minor}.$((patch + 1))"
next_minor="v${major}.$((minor + 1)).0"
next_major="v$((major + 1)).0.0"

echo ""
echo "🎯 Version options:"
echo "1) Patch: $next_patch (bug fixes)"
echo "2) Minor: $next_minor (new features)"  
echo "3) Major: $next_major (breaking changes)"
echo "4) Custom version"

read -p "Select [1-4]: " choice

case $choice in
    1) new_version="$next_patch" ;;
    2) new_version="$next_minor" ;;
    3) new_version="$next_major" ;;
    4) 
        read -p "Enter version (e.g., v2.3.0): " new_version
        if [[ ! "$new_version" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
            echo "❌ Invalid version format"
            exit 1
        fi
        ;;
    *) echo "❌ Invalid choice"; exit 1 ;;
esac

echo ""
echo -e "📋 Release checklist for ${GREEN}$new_version${NC}:"
echo ""

# Manual checklist
echo "📝 Manual steps to complete:"
echo "[ ] 1. Update VERSION file: echo '$new_version' > VERSION"
echo "[ ] 2. Update package.json version to: ${new_version#v}"
echo "[ ] 3. Update README.md latest release section"
echo "[ ] 4. Update CHANGELOG.md with release notes"
echo "[ ] 5. Commit changes: git add . && git commit -m 'chore: bump version to $new_version'"
echo "[ ] 6. Create tag: git tag -a $new_version -m 'Release $new_version'"
echo "[ ] 7. Push changes: git push origin master && git push origin $new_version"
echo "[ ] 8. Create GitHub release from draft"
echo "[ ] 9. Verify Docker image published: mitre/vulcan:$new_version"
echo "[ ] 10. Update deployment environments"

echo ""
echo "🔧 Quick commands:"
echo "# Update VERSION file"
echo "echo '$new_version' > VERSION"
echo ""
echo "# Update package.json (requires jq)"
echo "jq '.version = \"${new_version#v}\"' package.json > package.json.tmp && mv package.json.tmp package.json"
echo ""
echo "# Commit and tag"
echo "git add VERSION package.json README.md CHANGELOG.md"
echo "git commit -m 'chore: release $new_version'"
echo "git tag -a $new_version -m 'Release $new_version'"
echo "git push origin master && git push origin $new_version"

echo ""
echo -e "${YELLOW}💡 Tip: After the modernized release process (Issue #678), this will all be automated!${NC}"

Would you like me to create this current release helper script as a quick win for the team? It could be useful right now while we work toward the automated solution in Issue #678.

This provides immediate value and makes the current manual process less error-prone until we implement the full automation!

[aaronlippold]

Vulcan Release Process Analysis

Current State

📋 Release Workflow (from Wiki)

1. Automated Draft Creation: Every 14 days via GitHub Actions
2. Manual Review: Developer reviews/edits draft release
3. Version Management: Update multiple files manually
4. Testing: Automated test suite + manual staging verification
5. Publishing: Manual release publication

⚙️ Current Automation

• ✅ Draft Release Creation (.github/workflows/create-draft-release.yml)
  • Runs every 14 days automatically
  • Auto-calculates next patch version (v2.1.8 → v2.1.9)
  • Downloads source artifacts
  • Creates GitHub draft release
• ✅ Docker Publishing (.github/workflows/push-to-docker.yml)
  • Triggers on successful test suite OR published release
  • Publishes mitre/vulcan:latest on master pushes
  • Publishes mitre/vulcan:v2.x.x on release publication
  • Uses DockerHub secrets for authentication
• ✅ Test Automation (.github/workflows/run-tests.yml)
  • Runs on PRs, pushes, draft releases
  • PostgreSQL + LDAP container testing
  • RSpec + RuboCop + Yarn lint

📁 Version Management

• ❌ Manual Updates Required:
  • VERSION file (currently v2.1.8)
  • package.json version (currently 2.1.8)
  • README.md latest release section
  • CHANGELOG.md entries

Pain Points & Inefficiencies

🔴 Major Issues:

1. Multiple Manual File Updates: Developer must update 4+ files for each
   release
2. Version Sync Problems: VERSION, package.json, README easily get out of
   sync
3. Manual Changelog: No automated changelog generation
4. Semantic Versioning Gaps: Only auto-increments patch versions
5. Release Notes Manual: Completely manual process
6. No Pre-release Testing: No automated staging deployment validation

🟡 Minor Issues:

1. 14-day Schedule: Fixed schedule doesn't align with actual development
   pace
2. GitHub Token Dependency: Changelog generation requires personal tokens
3. Asset Naming: "source code(zip)" has inconsistent naming

Improvement Opportunities

🚀 High-Impact Improvements:

1. Automated Version Management

Use conventional commits to auto-determine version bump

Update all files atomically in one step

files_to_update:
- VERSION
- package.json
- README.md (latest release section)
2. Smart Release Triggers

Replace fixed 14-day schedule with:

triggers:
- manual workflow_dispatch
- when accumulating commits reach threshold
- when breaking changes detected via conventional commits
3. Automated Changelog Generation

Use GitHub API + conventional commits

Generate release notes automatically

Group by feat/fix/breaking/docs

4. Staging Deployment Validation

Auto-deploy to staging

Run integration tests

Health check validation

Manual approval gate

🎯 Recommended Implementation Plan:

Phase 1: Version Management Automation

• Create .github/workflows/bump-version.yml
• Use semantic-release or similar tool
• Atomic updates to all version files
• Auto-generate changelog from conventional commits

Phase 2: Smart Release Scheduling

• Replace cron trigger with intelligent detection
• Add manual workflow_dispatch for immediate releases
• Use commit analysis to determine version bump type

Phase 3: Enhanced Release Validation

• Add staging deployment step
• Automated health checks
• Integration test validation
• Manual approval gates

Phase 4: Complete Automation

• Auto-publish on successful validation
• Slack/email notifications
• Release artifact validation
• Rollback capabilities

💡 Tools & Technologies to Consider:

• semantic-release: Automated version management + changelog
• conventional-changelog: Generate changelogs from conventional commits
• release-please: Google's release automation tool
• GitHub CLI: Streamline GitHub operations
• act: Test GitHub Actions locally

📊 ROI Analysis:

• Current Manual Effort: ~2-3 hours per release
• With Automation: ~15 minutes per release
• Risk Reduction: Eliminates version sync issues
• Quality Improvement: Consistent release process

Immediate Actionable Steps

1. Create GitHub Issue: "Modernize Release Process Automation"
2. Start with Version Sync: Create script to update all version files
3. Implement Conventional Commits: Standardize commit message format
4. Add workflow_dispatch: Enable manual release triggers
5. Create staging deployment: Validate releases before publication

This analysis shows Vulcan has good foundational automation but
significant opportunities for streamlining manual steps and improving
release quality!

So if you were going to fully rebuild and fix the vulcan release process,
what would you do and how would it look?

⏺ Complete Vulcan Release Process Redesign

If I were to fully rebuild Vulcan's release process from scratch, here's
how I'd design a modern, automated, and reliable system:

🎯 Design Principles

1. Zero Manual File Updates - All version bumps automated
2. Conventional Commits Driven - Release decisions based on commit
   analysis
3. Multi-Environment Validation - Staging → Production pipeline
4. Rollback Ready - Quick recovery from failed releases
5. Container-First - Modern deployment patterns
6. Security-First - Automated vulnerability scanning

🏗️ New Architecture

1. Commit Standards & Analysis

  # .github/workflows/commit-analysis.yml
  name: Analyze Commits for Release Planning

  on:
    push:
      branches: [master]

  jobs:
    analyze:
      runs-on: ubuntu-latest
      outputs:
        should_release: ${{ steps.analysis.outputs.should_release }}
        version_bump: ${{ steps.analysis.outputs.version_bump }}
        release_notes: ${{ steps.analysis.outputs.release_notes }}

      steps:
        - uses: actions/checkout@v4
          with:
            fetch-depth: 0

        - name: Analyze commits since last release
          id: analysis
          uses: semantic-release/semantic-release@v22
          with:
            dry_run: true
          env:
            GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Conventional Commit Standards:
feat: add OIDC auto-discovery (#672) # → Minor version bump
fix: resolve LDAP authentication issue (#669) # → Patch version bump
feat!: migrate to rails-settings-cached # → Major version bump
docs: update installation guide # → No version bump
ci: update GitHub Actions versions # → No version bump

2. Automated Release Decision Engine

  # .github/workflows/release-orchestrator.yml
  name: Release Orchestrator

  on:
    workflow_run:
      workflows: ["Analyze Commits"]
      types: [completed]
      branches: [master]
    workflow_dispatch:
      inputs:
        release_type:
          description: 'Release type'
          required: true
          default: 'auto'
          type: choice
          options:
          - auto
          - patch
          - minor
          - major
          - prerelease

  jobs:
    release_decision:
      if: ${{ github.event.workflow_run.conclusion == 'success' ||
  github.event_name == 'workflow_dispatch' }}
      runs-on: ubuntu-latest
      outputs:
        should_release: ${{ steps.decision.outputs.should_release }}
        version_bump: ${{ steps.decision.outputs.version_bump }}

      steps:
        - name: Make release decision
          id: decision
          run: |
            if [[ "${{ github.event_name }}" == "workflow_dispatch" ]];
  then
              echo "should_release=true" >> $GITHUB_OUTPUT
              echo "version_bump=${{ github.event.inputs.release_type }}"
  >> $GITHUB_OUTPUT
            else
              # Use analysis from previous workflow
              echo "should_release=${{ needs.analyze.outputs.should_release
   }}" >> $GITHUB_OUTPUT
              echo "version_bump=${{ needs.analyze.outputs.version_bump }}"
   >> $GITHUB_OUTPUT
            fi

3. Automated Version Management

  # .github/workflows/version-management.yml
  name: Automated Version Management

  jobs:
    version_bump:
      runs-on: ubuntu-latest
      if: needs.release_decision.outputs.should_release == 'true'

      steps:
        - uses: actions/checkout@v4
          with:
            token: ${{ secrets.VULCAN_RELEASE_TOKEN }}

        - name: Calculate new version
          id: version
          uses: mathieudutour/github-tag-action@v6.1
          with:
            github_token: ${{ secrets.GITHUB_TOKEN }}
            default_bump: ${{ needs.release_decision.outputs.version_bump
  }}
            dry_run: true

        - name: Update all version files atomically
          run: |
            NEW_VERSION="${{ steps.version.outputs.new_version }}"

            # Update VERSION file
            echo "$NEW_VERSION" > VERSION

            # Update package.json
            jq ".version = \"${NEW_VERSION#v}\"" package.json >
  package.json.tmp
            mv package.json.tmp package.json

            # Update README.md latest release section
            sed -i "s/Latest Release: \[v[0-9]\+\.[0-9]\+\.[0-9]\+\]/Latest
   Release: [$NEW_VERSION]/" README.md
            sed -i
  "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:$NEW_VERSION/"
  README.md

            # Update any Helm charts, Docker configs, etc.
            find docs/ -name "*.yaml" -o -name "*.yml" | xargs sed -i
  "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:$NEW_VERSION/g"

        - name: Generate changelog
          id: changelog
          uses: conventional-changelog/standard-version@v9.5.0
          with:
            skip:
              commit: true
              tag: true
            releaseCommitMessageFormat: "chore(release): {{currentTag}}"

        - name: Commit version updates
          run: |
            git config --local user.email "action@github.com"
            git config --local user.name "GitHub Action"
            git add VERSION package.json README.md CHANGELOG.md docs/
            git commit -m "chore(release): bump version to ${{
  steps.version.outputs.new_version }}"
            git push

4. Multi-Stage Validation Pipeline

  # .github/workflows/release-validation.yml
  name: Release Validation Pipeline

  jobs:
    # Stage 1: Build & Test
    build_and_test:
      runs-on: ubuntu-latest
      strategy:
        matrix:
          ruby: ['3.0', '3.1', '3.2']
          node: ['16', '18', '20']
      steps:
        - name: Run comprehensive test suite
          run: |
            bundle exec rspec --format RspecJunitFormatter --out
  test-results.xml
            yarn lint
            bundle exec rubocop --format json --out rubocop-results.json

        - name: Security scanning
          uses: github/super-linter@v4

        - name: Vulnerability assessment
          uses: aquasecurity/trivy-action@master

    # Stage 2: Build Release Artifacts
    build_artifacts:
      needs: build_and_test
      runs-on: ubuntu-latest
      outputs:
        image_digest: ${{ steps.build.outputs.digest }}

      steps:
        - name: Build and push Docker image
          id: build
          uses: docker/build-push-action@v5
          with:
            push: true
            tags: |
              mitre/vulcan:${{ needs.version_bump.outputs.version }}
              mitre/vulcan:latest
            platforms: linux/amd64,linux/arm64
            cache-from: type=gha
            cache-to: type=gha,mode=max

        - name: Generate SBOM
          uses: anchore/sbom-action@v0
          with:
            image: mitre/vulcan:${{ needs.version_bump.outputs.version }}

    # Stage 3: Staging Deployment
    deploy_staging:
      needs: build_artifacts
      runs-on: ubuntu-latest
      environment: staging

      steps:
        - name: Deploy to staging
          run: |
            # Deploy to staging K8s/Heroku/etc
            kubectl set image deployment/vulcan vulcan=mitre/vulcan:${{
  needs.version_bump.outputs.version }}
            kubectl rollout status deployment/vulcan

        - name: Run staging validation
          run: |
            # Health checks
            curl -f https://vulcan-staging.mitre.org/health

            # Integration tests
            bundle exec rspec spec/integration/

            # Performance tests
            ./bin/performance-test https://vulcan-staging.mitre.org

        - name: Security validation
          run: |
            # OWASP ZAP scan
            docker run -t owasp/zap2docker-stable zap-baseline.py -t
  https://vulcan-staging.mitre.org

    # Stage 4: Manual Approval Gate
    approval_gate:
      needs: deploy_staging
      runs-on: ubuntu-latest
      environment: production-approval

      steps:
        - name: Request manual approval
          uses: trstringer/manual-approval@v1
          with:
            secret: ${{ secrets.GITHUB_TOKEN }}
            approvers: rlakey,aaronlippold
            minimum-approvals: 1
            issue-title: "Release ${{ needs.version_bump.outputs.version }}
   approval"
            issue-body: |
              **Release**: ${{ needs.version_bump.outputs.version }}
              **Staging URL**: https://vulcan-staging.mitre.org
              **Changes**: ${{ needs.version_bump.outputs.changelog }}

              Please review staging deployment and approve for production
  release.

5. Production Release & Monitoring

  # .github/workflows/production-release.yml
  name: Production Release

  jobs:
    create_release:
      needs: approval_gate
      runs-on: ubuntu-latest

      steps:
        - name: Create GitHub release
          uses: softprops/action-gh-release@v1
          with:
            tag_name: ${{ needs.version_bump.outputs.version }}
            name: "Vulcan ${{ needs.version_bump.outputs.version }}"
            body: ${{ needs.version_bump.outputs.changelog }}
            files: |
              CHANGELOG.md
              docs/MIGRATION.md
            prerelease: ${{ contains(needs.version_bump.outputs.version,
  '-') }}

        - name: Update documentation site
          run: |
            # Update vulcan.mitre.org with new release
            curl -X POST "https://api.netlify.com/build_hooks/${{
  secrets.NETLIFY_BUILD_HOOK }}"

    deploy_production:
      needs: create_release
      runs-on: ubuntu-latest
      environment: production

      steps:
        - name: Deploy to production
          run: |
            # Production deployment (blue-green, canary, etc.)
            # This would be environment-specific

        - name: Post-deployment validation
          run: |
            # Health checks
            curl -f https://vulcan.mitre.org/health

            # Smoke tests
            ./bin/smoke-test https://vulcan.mitre.org

    notify_stakeholders:
      needs: deploy_production
      runs-on: ubuntu-latest

      steps:
        - name: Slack notification
          uses: 8398a7/action-slack@v3
          with:
            status: success
            text: |
              🚀 Vulcan ${{ needs.version_bump.outputs.version }} released
  successfully!

              📋 **Release Notes**:
  https://github.com/mitre/vulcan/releases/tag/${{
  needs.version_bump.outputs.version }}
              🌐 **Production**: https://vulcan.mitre.org
              📊 **Monitoring**: https://grafana.mitre.org/vulcan

        - name: Email notification
          uses: dawidd6/action-send-mail@v3
          with:
            to: vulcan-team@mitre.org
            subject: "Vulcan ${{ needs.version_bump.outputs.version }}
  Released"
            body: |
              The new Vulcan release is now live in production.

              Release: ${{ needs.version_bump.outputs.version }}
              Changes: ${{ needs.version_bump.outputs.changelog }}

6. Rollback & Recovery System

  # .github/workflows/emergency-rollback.yml
  name: Emergency Rollback

  on:
    workflow_dispatch:
      inputs:
        target_version:
          description: 'Version to rollback to'
          required: true
        reason:
          description: 'Rollback reason'
          required: true

  jobs:
    rollback:
      runs-on: ubuntu-latest
      environment: production-emergency

      steps:
        - name: Validate rollback target
          run: |
            # Ensure target version exists and is valid
            git tag | grep -q "${{ github.event.inputs.target_version }}"

        - name: Execute rollback
          run: |
            # Rollback deployment
            kubectl rollout undo deployment/vulcan --to-revision=$(kubectl
  rollout history deployment/vulcan | grep ${{
  github.event.inputs.target_version }} | awk '{print $1}')

            # Rollback database if needed (with safety checks)
            if [[ "${{ github.event.inputs.include_database }}" == "true"
  ]]; then
              # Database rollback logic with extensive safety checks
              echo "Database rollback would require manual intervention"
            fi

        - name: Validate rollback
          run: |
            kubectl rollout status deployment/vulcan
            curl -f https://vulcan.mitre.org/health

        - name: Create incident report
          uses: actions/github-script@v7
          with:
            script: |
              await github.rest.issues.create({
                ...context.repo,
                title: `🚨 Emergency Rollback:
  ${context.payload.inputs.target_version}`,
                body: `
                **Rollback Details:**
                - **From**: Latest
                - **To**: ${context.payload.inputs.target_version}
                - **Reason**: ${context.payload.inputs.reason}
                - **Executed by**: @${context.actor}
                - **Time**: ${new Date().toISOString()}

                **Next Steps:**
                - [ ] Investigate root cause
                - [ ] Fix underlying issue
                - [ ] Plan new release
                - [ ] Update monitoring/alerting
                `,
                labels: ['incident', 'rollback', 'priority-high']
              });

📊 Configuration Files

Release Configuration

  // .releaserc.json
  {
    "branches": ["master"],
    "plugins": [
      "@semantic-release/commit-analyzer",
      "@semantic-release/release-notes-generator",
      "@semantic-release/changelog",
      [
        "@semantic-release/exec",
        {
          "prepareCmd": "scripts/update-version-files.sh
  ${nextRelease.version}"
        }
      ],
      "@semantic-release/github",
      [
        "@semantic-release/git",
        {
          "assets": ["VERSION", "package.json", "CHANGELOG.md",
  "README.md"],
          "message": "chore(release): ${nextRelease.version} [skip
  ci]\n\n${nextRelease.notes}"
        }
      ]
    ]
  }

Version Update Script

  #!/bin/bash
  # scripts/update-version-files.sh

  NEW_VERSION=$1

  # Update VERSION file
  echo "v$NEW_VERSION" > VERSION

  # Update package.json
  jq ".version = \"$NEW_VERSION\"" package.json > package.json.tmp
  mv package.json.tmp package.json

  # Update README.md
  sed -i "s/Latest Release: \[v[0-9]\+\.[0-9]\+\.[0-9]\+\]/Latest Release:
  [v$NEW_VERSION]/" README.md
  sed -i
  "s/mitre\/vulcan:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:v$NEW_VERSION/"
   README.md

  # Update documentation
  find docs/ -name "*.yaml" -o -name "*.yml" | xargs sed -i "s/mitre\/vulca
  n:v[0-9]\+\.[0-9]\+\.[0-9]\+/mitre\/vulcan:v$NEW_VERSION/g"

  echo "✅ Updated all version files to v$NEW_VERSION"

🎯 Benefits of New System

🚀 Automation Benefits:

• Zero Manual Steps: Complete automation from commit → production
• Consistent Quality: Every release follows same validation pipeline
• Fast Recovery: Automated rollback in <5 minutes
• Security First: Automated vulnerability scanning & validation

📈 Developer Experience:

• Simple: git commit -m "feat: new feature" → automatic release
• Predictable: Clear release timeline and process
• Safe: Staging validation + approval gates
• Transparent: Full audit trail and notifications

🔒 Enterprise Ready:

• Compliance: Full audit trail and approval workflows
• Security: Automated scanning and validation
• Reliability: Multi-stage validation and rollback capabilities
• Monitoring: Comprehensive notifications and metrics

💰 Cost Savings:

• Time: 3 hours → 15 minutes per release
• Risk: Eliminate human error in version management
• Quality: Consistent testing and validation
• Recovery: Fast rollback vs lengthy incident response

This system transforms Vulcan releases from a manual, error-prone process
into a modern, automated pipeline that rivals the best practices of
major tech companies!