Skip to content

Latest commit

 

History

History
630 lines (477 loc) · 18 KB

File metadata and controls

630 lines (477 loc) · 18 KB

Objeck Language CI/CD Architecture

Version: 1.1 Last Updated: 2026-04-05

This document describes the technical architecture of the Objeck Language continuous integration and continuous deployment (CI/CD) system.

📋 Table of Contents

  1. Overview
  2. Architecture
  3. Workflows
  4. Secrets Management
  5. Build Matrix
  6. Caching Strategy
  7. Security
  8. Cost Analysis
  9. Maintenance

Overview

The Objeck Language CI/CD system is built entirely on GitHub Actions, providing:

  • Zero cost (public repository = unlimited build minutes)
  • Parallel builds across 6 platforms
  • Automated code signing (no manual intervention)
  • Multi-destination distribution (GitHub, Sourceforge, objeck.org)
  • Full release automation (60 minutes end-to-end)

Design Goals

  1. Eliminate manual work - One git tag triggers everything
  2. Minimize build time - Parallel builds + aggressive caching
  3. Maximize reliability - Automated testing at every step
  4. Zero cost - Leverage free GitHub Actions for public repos
  5. Easy maintenance - Clear separation of concerns, modular design

Architecture

Three-Tier Workflow Structure

┌─────────────────────────────────────────────────────────────────┐
│                    CI Build (ci-build.yml)                      │
│  Trigger: Every push/PR to master                               │
│  Purpose: Fast feedback, catch issues early                     │
│  Duration: ~20 minutes (with cache)                             │
│  Platforms: Windows x64, Linux x64/ARM64, macOS ARM64           │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ Tag pushed (v*.*.*)
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                Release Build (release-build.yml)                │
│  Trigger: Git tag (v2026.2.1) or manual                         │
│  Purpose: Full production builds with installers                │
│  Duration: ~45 minutes (parallel)                               │
│  Platforms: Windows x64/ARM64, Linux x64/ARM64, macOS ARM64, LSP│
│  Outputs: MSI, ZIP, TGZ, API docs                               │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ Manual trigger (with run_id)
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│              Release Publish (release-publish.yml)              │
│  Trigger: Manual dispatch                                       │
│  Purpose: Sign, rename, distribute                              │
│  Duration: ~15 minutes                                          │
│  Actions: Code signing, GitHub Release, Sourceforge, docs deploy│
└─────────────────────────────────────────────────────────────────┘

Why Manual Trigger for Publish?

The release publish workflow requires manual triggering to:

  1. Review build artifacts before distribution
  2. Verify all platform builds completed successfully
  3. Allow testing installers before public release
  4. Provide control over release timing

Future enhancement: Could be fully automatic with approval gates.


Workflows

1. CI Build (ci-build.yml)

Purpose: Validate every code change

Triggers:

  • Push to master branch
  • Pull requests targeting master

Jobs:

  1. Build Matrix (parallel):

    • Windows x64
    • Windows ARM64
    • Linux x64
    • Linux ARM64
    • macOS ARM64
  2. For Each Platform:

    • Install dependencies (cached)
    • Build compiler bootstrap
    • Build full toolchain
    • Run test suite
    • Run regression tests
    • Upload artifacts (7-day retention)
  3. Linux x64 Only:

    • Generate API documentation
    • Upload API docs artifact
  4. Status Job:

    • Aggregate results
    • Report overall CI status

Optimization:

  • Dependency caching (~80% cache hit rate)
  • ccache for C++ compilation (~60% speedup)
  • Parallel job execution (5 concurrent)

Fast Feedback:

  • Typical time: 15-20 minutes (cached)
  • Worst case: 30 minutes (cold cache)

2. Release Build (release-build.yml)

Purpose: Create production-ready release artifacts

Triggers:

  • Git tag matching v*.*.* (automatic)
  • Manual workflow dispatch (for testing)

Jobs:

  1. Prepare Job:

    • Extract version from tag
    • Update version files automatically
    • Generate version metadata artifact
  2. Build Matrix (parallel):

    • Windows x64 (MSI + ZIP)
    • Windows ARM64 (MSI + ZIP)
    • Linux x64 (TGZ)
    • Linux ARM64 (TGZ)
    • macOS ARM64 (TGZ)
  3. Build LSP Job:

    • Package Language Server Protocol binaries
    • Create LSP ZIP archive
  4. Build Docs Job:

    • Generate API documentation with correct version
    • Package as ZIP
  5. Summary Job:

    • Aggregate build results
    • Generate build summary report
    • Notify on failures

Key Features:

  • Automatic versioning from git tags (no manual edits)
  • Artifact retention for 7 days
  • Unsigned installers (signing happens in publish step)

Duration: ~45 minutes (parallel builds)


3. Release Publish (release-publish.yml)

Purpose: Sign, rename, and distribute release

Triggers:

  • Manual workflow dispatch only

Required Inputs:

  • version: Version number (e.g., 2026.2.1)
  • run_id: Release build workflow run ID
  • skip_sourceforge: Optional flag
  • skip_docs_deploy: Optional flag

Jobs:

  1. Prepare Job:

    • Download all build artifacts
    • Build binary renaming tool
    • Rename files with version numbers
    • Decode code signing certificate (if available)
    • Sign Windows MSI installers
    • Verify signatures
    • Cleanup certificate securely
    • Upload final artifacts (30-day retention)
  2. GitHub Release Job:

    • Generate release notes
    • Create GitHub Release
    • Upload all binaries
    • Tag association
  3. Sourceforge Upload Job:

    • Setup SSH authentication
    • Create version directory
    • Upload via SFTP
    • Cleanup SSH keys
  4. Deploy Docs Job:

    • Extract API documentation
    • Upload to objeck.org via rsync
    • Update 'latest' symlink
    • Cleanup SSH keys
  5. Summary Job:

    • Aggregate deployment status
    • Generate release summary
    • Provide download links

Security:

  • Secrets never logged
  • Temporary files cleaned up (always)
  • SSH keys removed after use

Duration: ~15 minutes


Secrets Management

Required Secrets

Set in: Repository Settings → Secrets and variables → Actions

Secret Purpose Format Expiration
APPLE_CERTIFICATE_BASE64 macOS Developer ID Application cert Base64-encoded P12 5 years
APPLE_CERTIFICATE_PASSWORD Application cert password Plain text Same as cert
APPLE_INSTALLER_CERT_BASE64 macOS Developer ID Installer cert Base64-encoded P12 5 years
APPLE_INSTALLER_CERT_PASSWORD Installer cert password Plain text Same as cert
KEYCHAIN_PASSWORD CI temporary keychain Plain text N/A
APPLE_ID Apple ID for notarization Email N/A
APPLE_TEAM_ID Apple Developer Team ID 10-char string N/A
APPLE_APP_PASSWORD App-specific password for notarization Plain text Revocable
CODESIGN_CERT_BASE64 Windows Sectigo code signing cert Base64-encoded PFX 1-3 years
CODESIGN_PASSWORD Windows cert password Plain text Same as cert
SOURCEFORGE_SSH_KEY Sourceforge SFTP SSH private key Rotate every 2 years
SOURCEFORGE_USERNAME Sourceforge account Username N/A
OBJECK_ORG_SSH_KEY Web server access SSH private key Rotate every 2 years
OBJECK_ORG_USER Web server username Username N/A

Creating Secrets

macOS Code Signing (see SIGNING.md for full details):

# Export Developer ID cert from Keychain Access as .p12
# Base64 encode and set as GitHub secret
base64 -i certificate.p12 | gh secret set APPLE_CERTIFICATE_BASE64
# CI workflow creates temporary keychain and imports cert automatically

Windows Code Signing Certificate:

# Convert PFX to Base64
certutil -encode certificate.pfx certificate.base64.txt

# Copy content to CODESIGN_CERT_BASE64 secret (remove header/footer)

# Set CODESIGN_PASSWORD to your certificate password

SSH Keys:

# Generate SSH key pair
ssh-keygen -t ed25519 -f objeck_sf_key -N ""

# Add public key to Sourceforge account (Settings → SSH Keys)

# Copy private key content to SOURCEFORGE_SSH_KEY secret
cat objeck_sf_key

# Set SOURCEFORGE_USERNAME to your Sourceforge username

# Repeat for objeck.org with OBJECK_ORG_SSH_KEY and OBJECK_ORG_USER

Security Best Practices

  1. Never commit secrets to repository
  2. Use GitHub Secrets (encrypted at rest, masked in logs)
  3. Rotate regularly (SSH keys every 2 years, certificates before expiry)
  4. Limit scope (use dedicated keys, not personal accounts)
  5. Monitor usage (GitHub audit logs)

Build Matrix

Platform Configuration

Platform Runner OS Version Arch Build Time Artifacts
Windows x64 windows-latest Server 2022 x64 25 min MSI, ZIP
Windows ARM64 windows-latest Server 2022 ARM64 28 min MSI, ZIP
Linux x64 ubuntu-latest Ubuntu 22.04 x64 20 min TGZ
Linux ARM64 ubuntu-24.04-arm Ubuntu 24.04 ARM64 22 min TGZ
macOS ARM64 macos-14 macOS 14 ARM64 30 min TGZ
LSP windows-latest Server 2022 x64 10 min ZIP

Notes:

  • All builds run in parallel (maximum parallelization)
  • Total elapsed time = longest single build (~30 minutes)
  • Windows ARM64 is cross-compiled on x64 runner

Runner Specifications

GitHub-hosted runners (standard):

  • CPU: 2-4 cores
  • RAM: 7-14 GB
  • Disk: 14-150 GB SSD
  • Network: Fast (Azure datacenter)

Cost: $0 (unlimited for public repos)


Caching Strategy

Dependency Caching

Linux (APT):

uses: actions/cache@v4
with:
  path: /var/cache/apt/archives
  key: ${{ runner.os }}-${{ matrix.arch }}-apt-${{ hashFiles('.github/**') }}
  restore-keys: |
    ${{ runner.os }}-${{ matrix.arch }}-apt-

macOS (Homebrew):

uses: actions/cache@v4
with:
  path: |
    ~/Library/Caches/Homebrew
    /opt/homebrew/Cellar
  key: ${{ runner.os }}-brew-${{ hashFiles('.github/**') }}
  restore-keys: |
    ${{ runner.os }}-brew-

ccache (Compilation):

uses: actions/cache@v4
with:
  path: ~/.ccache
  key: ${{ runner.os }}-${{ matrix.arch }}-ccache-${{ github.sha }}
  restore-keys: |
    ${{ runner.os }}-${{ matrix.arch }}-ccache-

Cache Performance

Cache Type Hit Rate Speedup Size
APT packages ~80% 2-3 min saved 100-200 MB
Homebrew ~75% 5-8 min saved 200-500 MB
ccache ~60% 5-10 min saved 200-400 MB

Total cache usage: ~500 MB per platform (well within 500 MB limit)

Cache Invalidation

Caches are invalidated when:

  • Workflow files change (.github/** hash changes)
  • Manual cache clear (repository settings)
  • 7 days of inactivity (GitHub auto-expires)

Security

Code Signing

Windows MSI Signing:

  1. Certificate stored as Base64-encoded secret
  2. Decoded to temporary file during workflow
  3. Used with signtool for signing
  4. Always cleaned up (even on failure):
    - name: Cleanup certificate
      if: always()
      run: Remove-Item cert.pfx -Force
  5. Signatures timestamped (valid after cert expires)

Verification:

signtool verify /pa setup.msi

SSH Key Management

Best Practices:

  1. Dedicated keys (not personal accounts)
  2. Ed25519 algorithm (modern, secure)
  3. No passphrase (can't be interactive in CI)
  4. Restricted permissions (chmod 600)
  5. Always cleaned up after use
  6. Added to known_hosts (prevent MITM)

Example:

- name: Setup SSH key
  run: |
    mkdir -p ~/.ssh
    echo "${{ secrets.SOURCEFORGE_SSH_KEY }}" > ~/.ssh/sf_key
    chmod 600 ~/.ssh/sf_key
    ssh-keyscan frs.sourceforge.net >> ~/.ssh/known_hosts

- name: Cleanup SSH key
  if: always()
  run: rm -f ~/.ssh/sf_key

Secrets in Logs

GitHub Actions automatically masks secret values in logs:

  • echo ${{ secrets.MY_SECRET }}***
  • Certificate passwords, SSH keys, etc. are never visible

Dependency Security

Mitigation:

  • Pin GitHub Actions to commit SHAs (not tags)
  • Use official actions only (e.g., actions/checkout@v4)
  • Regularly update action versions
  • Monitor GitHub Security Advisories

Cost Analysis

Current Cost: $0/month ✅

Why Free?

  • Objeck is a public repository
  • GitHub Actions is unlimited for public repos
  • Includes:
    • Unlimited build minutes (Linux, Windows, macOS)
    • 500 MB artifact storage (sufficient with 7-day retention)
    • 20 concurrent jobs

If Repository Becomes Private

Estimated Cost: $15-25/month

Runner Cost/minute Minutes/month Monthly Cost
Linux $0.008 500 $4
Windows $0.016 400 $6.40
macOS $0.08 100 $8
Total ~$18.40

Assumptions:

  • 10 releases/month
  • 5 CI builds/day
  • Average build time: 20 minutes

Still within budget: Target was $20-$50/month.


Maintenance

Regular Tasks

Weekly:

  • Monitor build times (should stay under 30 minutes cached)
  • Check cache hit rates (aim for >75%)
  • Review failed builds

Monthly:

  • Review secret expiration dates
  • Update GitHub Actions versions
  • Check for security advisories

Yearly:

  • Rotate SSH keys (every 2 years)
  • Renew code signing certificate (before expiry)
  • Review and optimize caching strategy

Updating Workflows

Best Practices:

  1. Test changes in a branch first
  2. Use workflow_dispatch for manual testing
  3. Monitor CI builds before tagging release
  4. Keep workflows simple and well-commented
  5. Use composite actions to reduce duplication

Testing Workflow Changes:

# Push changes to a branch
git checkout -b test-ci-update
git add .github/workflows/
git commit -m "Test CI workflow update"
git push origin test-ci-update

# Create PR to trigger CI
gh pr create --title "Test CI update" --body "Testing workflow changes"

# Monitor results
gh pr checks --watch

# If successful, merge to master
gh pr merge --squash

Monitoring

GitHub Actions Dashboard:

Command Line:

# List recent runs
gh run list --limit 10

# Watch a running build
gh run watch

# View logs for a specific run
gh run view <run-id> --log

# Re-run failed jobs
gh run rerun <run-id> --failed

Status Badges: Add to README.md:

[![CI Build](https://github.com/objeck/objeck-lang/actions/workflows/ci-build.yml/badge.svg)](https://github.com/objeck/objeck-lang/actions/workflows/ci-build.yml)

Troubleshooting

Common Issues:

  1. Cache corruption

    • Symptom: Build failures after successful run
    • Solution: Clear caches in repository settings
  2. Runner out of disk space

    • Symptom: "No space left on device"
    • Solution: Clean up build artifacts during workflow
  3. Dependency installation timeout

    • Symptom: APT/Homebrew hangs
    • Solution: Add timeout-minutes to steps
  4. SSH connection failures

    • Symptom: "Permission denied" or "Connection refused"
    • Solution: Verify SSH keys, check server availability

Getting Help:


Future Enhancements

Potential Improvements

  1. Automatic Release Publishing

    • Add approval gates instead of manual trigger
    • Publish immediately after successful build
  2. Continuous Deployment

    • Deploy to test environment on every commit
    • Automatic nightly builds
  3. Enhanced Testing

    • Performance benchmarks
    • Memory leak detection
    • Security scanning (CodeQL)
  4. Artifact Signing

    • Sign Linux/macOS binaries (GPG)
    • Add checksums (SHA256)
  5. Docker Images

    • Publish Docker images to Docker Hub
    • Multi-architecture support
  6. Release Notes Automation

    • Generate from commit messages
    • Categorize changes (features, fixes, etc.)
  7. Notifications

    • Slack/Discord notifications
    • Email on release completion

References


Changelog

Version Date Changes
1.0 2026-02-10 Initial CI/CD architecture documentation