CI/CD Testing Guide

This document covers continuous integration testing configuration, pipeline stages, and optimizing test runs in CI environments.

Overview
GitHub Actions Workflow
Pipeline Stages
cargo-nextest Profiles
Environment Variables
Test Artifacts
Local CI Emulation
Troubleshooting
Performance Optimization

Overview

Tests run automatically on:

Every pull request to main or develop
Every push to main or develop

Workflow file: .github/workflows/ci.yml

CI Strategy

Parallel execution: Backend and frontend tests run concurrently
Dual database testing: SQLite for fast unit tests, PostgreSQL for integration tests
Smart caching: sccache and dependency caching reduce build time by 30-50%
Coverage on PRs: Coverage reports generated only for pull requests

GitHub Actions Workflow

Trigger Events

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

Job Dependencies

┌─────────────────┐     ┌─────────────────────┐
│  backend-lint   │     │   frontend-lint     │
└────────┬────────┘     └──────────┬──────────┘
         │                         │
         ▼                         ▼
┌─────────────────┐     ┌─────────────────────┐
│backend-unit-test│     │   frontend-test     │
└────────┬────────┘     └──────────┬──────────┘
         │                         │
         ▼                         │
┌─────────────────────────┐        │
│backend-integration-test │        │
└────────┬────────────────┘        │
         │                         │
         ▼                         ▼
┌─────────────────────────────────────────────┐
│              backend-build                   │
└──────────────────────┬──────────────────────┘
                       │
                       ▼
              ┌─────────────────┐
              │  docker-build   │
              └─────────────────┘

Pipeline Stages

1. Backend Validation (Parallel)

backend-lint:
  - cargo fmt --check
  - cargo clippy --all-targets --all-features -- -D warnings

backend-security:
  - cargo audit
  - cargo deny check

2. Backend Tests (Parallel, after lint)

Unit Tests (SQLite)

backend-unit-test:
  database: sqlite::memory:
  runner: cargo nextest
  threads: 4
  retries: 2
  timeout: 10 minutes

Fast feedback with in-memory SQLite. Tests that require PostgreSQL features are automatically skipped.

Integration Tests (PostgreSQL)

backend-integration-test:
  service: postgres:16-alpine
  database: postgres://ampel:ampel@localhost:5432/ampel_test
  runner: cargo nextest
  threads: 2
  retries: 3
  timeout: 15 minutes

Comprehensive testing with real PostgreSQL. Each test gets an isolated database instance.

3. Frontend Tests (Parallel, after lint)

frontend-lint:
  - pnpm run lint
  - pnpm run type-check

frontend-test:
  runner: vitest
  coverage: true
  artifacts: coverage report (7 days)

4. Build (After tests pass)

backend-build:
  - cargo build --release

frontend-build:
  - pnpm run build

docker-build:
  - docker build (push events only)

5. Coverage (PRs only)

backend-coverage:
  tool: cargo-llvm-cov
  database: PostgreSQL (required)
  upload: Codecov

cargo-nextest Profiles

Configuration in .config/nextest.toml:

fast (Local Development)

[profile.fast]
test-threads = 8
retries = 0
fail-fast = true
slow-timeout = { period = "30s", terminate-after = 2 }

When: Quick feedback during coding
Speed: Fastest
Command: cargo nextest run --profile fast

default (Standard Testing)

[profile.default]
test-threads = 4
retries = 2
fail-fast = false
slow-timeout = { period = "60s", terminate-after = 3 }

When: Pre-commit checks
Speed: Moderate
Command: cargo nextest run

ci (GitHub Actions)

[profile.ci]
test-threads = 4
retries = 3
fail-fast = false
slow-timeout = { period = "120s", terminate-after = 3 }

When: Automatically in CI
Speed: Optimized for reliability
Command: cargo nextest run --profile ci

coverage (Test Coverage)

[profile.coverage]
test-threads = 1
retries = 0

When: Generating coverage reports
Speed: Slower (single-threaded for accurate coverage)
Command: cargo nextest run --profile coverage

Environment Variables

CI Environment

# Rust build
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
RUSTFLAGS: '-D warnings'
RUST_VERSION: '1.92.0'

# Test configuration
DATABASE_URL: postgres://ampel:ampel@localhost:5432/ampel_test
TEST_DATABASE_TYPE: postgres
JWT_SECRET: test-jwt-secret-for-ci-minimum-32-chars
ENCRYPTION_KEY: dGVzdC1lbmNyeXB0aW9uLWtleS0zMi1ieXRlcw==
RUST_LOG: info
CI: true

# Performance
SCCACHE_GHA_ENABLED: 'true'
RUSTC_WRAPPER: 'sccache'

Required for All Tests

JWT_SECRET=test-jwt-secret-for-ci-minimum-32-chars
ENCRYPTION_KEY=dGVzdC1lbmNyeXB0aW9uLWtleS0zMi1ieXRlcw==

Database Selection

# SQLite (fast unit tests)
DATABASE_URL=sqlite::memory:
TEST_DATABASE_TYPE=sqlite

# PostgreSQL (integration tests)
DATABASE_URL=postgres://ampel:ampel@localhost:5432/ampel_test
TEST_DATABASE_TYPE=postgres

Test Artifacts

Uploaded on test completion with 7-day retention:

Artifact	Contents
`backend-unit-test-results`	nextest JUnit output
`backend-integration-test-results`	nextest JUnit output
`frontend-coverage`	Vitest coverage report
`backend-coverage-report`	Tarpaulin coverage report

Downloading Artifacts

# Using GitHub CLI
gh run download <run-id> -n backend-unit-test-results

Local CI Emulation

Run Same Tests as CI

# Backend unit tests (SQLite)
DATABASE_URL=sqlite::memory: cargo nextest run --profile ci

# Backend integration tests (PostgreSQL)
docker compose up -d postgres
DATABASE_URL=postgres://ampel:ampel@localhost:5432/ampel_test \
  cargo nextest run --profile ci

# Frontend tests
cd frontend && pnpm test -- --run --coverage

Full CI Check

# All checks in one command
make ci

Before Committing Checklist

# 1. Run fast unit tests
cargo nextest run --profile fast

# 2. Run linter
make lint-backend

# 3. Check formatting
make format-check-backend

Before Submitting PR Checklist

# 1. Start PostgreSQL
docker compose up -d postgres

# 2. Run full test suite
make test-backend

# 3. Generate coverage
cargo llvm-cov --all-features --workspace --html --output-dir coverage

# 4. Run all CI checks
make ci

Troubleshooting

Tests Pass Locally but Fail in CI

Likely causes:

Using different databases (SQLite vs PostgreSQL)
Different environment variables
Timing issues / race conditions

Solutions:

# Test with same environment as CI
DATABASE_URL=sqlite::memory: cargo nextest run --profile ci

# Then test with PostgreSQL
DATABASE_URL=postgres://... cargo nextest run --profile ci

Slow Test Performance

Check which tests are slow:

cargo nextest run --profile ci
# Look for "SLOW" markers in output

Speed up tests:

# Use more threads
cargo nextest run --test-threads 8

# Use fast profile
cargo nextest run --profile fast

Database Connection Errors

PostgreSQL not running:

# Start PostgreSQL
docker compose up -d postgres

# Wait for it to be ready
until pg_isready -h localhost -p 5432 -U ampel; do sleep 1; done

SQLite permission errors:

# Use in-memory SQLite
DATABASE_URL=sqlite::memory: cargo nextest run

Flaky Tests

Run with retries:

# 2 retries (default profile)
cargo nextest run

# 3 retries (CI profile)
cargo nextest run --profile ci

# Custom retries
cargo nextest run --retries 5

Viewing CI Results

# View recent workflow runs
gh run list

# Watch current workflow
gh run watch

# View specific run
gh run view <run-id>

# View CI status for current branch
gh pr checks

Performance Optimization

Build Caching

CI uses sccache for faster builds:

SCCACHE_GHA_ENABLED: 'true'
RUSTC_WRAPPER: 'sccache'

This reduces rebuild time by 30-50%.

Dependency Caching

- uses: Swatinem/rust-cache@v2
  with:
    shared-key: 'ci'

Parallel Test Execution

Backend unit and integration tests run in parallel
Frontend tests run in parallel with backend
Up to 4 test threads per job

Smart Test Selection (Future)

Consider implementing:

Run only tests affected by changed files
Skip unchanged crates
Incremental test runs

Common Workflows

Quick Validation During Development

cargo nextest run --profile fast

Speed: ~2 seconds for 100 tests

Pre-commit Validation

make ci-backend

Runs lint, format check, and tests

Full Validation Before PR

docker compose up -d postgres
make test-backend
cargo tarpaulin --all-features --workspace

Speed: ~30-60 seconds

CI Emulation

cargo nextest run --profile ci

Same configuration as GitHub Actions

Summary

Use Case	Command	Speed
Quick check	`cargo nextest run --profile fast`	2s
Pre-commit	`make test-backend`	5s
Full validation	`docker compose up -d && cargo nextest run`	30-60s
Coverage	`cargo llvm-cov --all-features --workspace`	30-60s
CI emulation	`cargo nextest run --profile ci`	30-60s

FilesExpand file tree

CI.md

Latest commit

History