12 Development Guide

Development Guide

Relevant source files

The following files were used as context for generating this wiki page:

Purpose and Scope

This document provides an overview of the development workflow, CI/CD infrastructure, testing strategy, and code ownership model for contributors and maintainers working on ZeroClaw. It covers the practical mechanics of submitting changes, understanding automated checks, and navigating the review process.

For detailed PR submission requirements and review contracts, see Contributing. For comprehensive CI/CD workflow documentation, see CI/CD Workflows. For testing patterns and test execution, see Testing. For ownership structure and review routing, see Code Ownership.

Development Workflow Overview

ZeroClaw follows a risk-based, high-throughput PR workflow designed to maintain quality under heavy contribution volume while keeping merge velocity predictable. The workflow uses automated triage, label-based routing, and tiered review depth based on risk classification.

PR Lifecycle Stages

stateDiagram-v2
    [*] --> Intake
    Intake --> Validation: "Template complete"
    Validation --> FastReview: "Low risk + CI green"
    Validation --> DeepReview: "High risk paths"
    FastReview --> ReadyToMerge: "Approved"
    DeepReview --> ReadyToMerge: "Security + rollback validated"
    ReadyToMerge --> Merged
    Merged --> [*]
    
    Validation --> IntakeBlocked: "CI failing"
    IntakeBlocked --> Validation: "Fixed"
    DeepReview --> IntakeBlocked: "Missing evidence"

Sources: docs/pr-workflow.md:108-138

Key Workflow Characteristics

Stage	Automation	Human Decision
Intake	Auto-label by path/size/risk, apply contributor tiers	Template completion check
Validation	CI Required Gate (lint, test, build)	N/A - deterministic
Review	Route by risk labels, first-time guidance	Maintainer approval, CODEOWNERS review
Merge	None	Squash merge with rollback verification

Contributor Tier Thresholds:

trusted: ≥5 merged PRs
experienced: ≥10 merged PRs
principal: ≥20 merged PRs
distinguished: ≥50 merged PRs

Sources: docs/pr-workflow.md:113-119, docs/pr-workflow.md:47-48

CI/CD Architecture

The CI/CD system uses GitHub Actions with a layered validation strategy: merge-blocking checks remain small and deterministic, while optional checks handle automation and maintenance without blocking development.

CI Workflow Integration Map

graph TB
    PR["Pull Request"] --> IntakeChecks["pr-intake-checks.yml<br/>Template + Tabs + Whitespace"]
    PR --> Labeler["pr-labeler.yml<br/>Scope/Size/Risk/Module"]
    PR --> AutoResponse["pr-auto-response.yml<br/>First-time guidance"]
    
    IntakeChecks --> CIGate["ci-run.yml<br/>CI Required Gate"]
    Labeler --> CIGate
    
    CIGate --> LintGates["lint-strict-delta job<br/>rust_strict_delta_gate.sh"]
    CIGate --> DocsGates["docs-quality job<br/>docs_quality_gate.sh + link check"]
    CIGate --> TestBuild["test + build jobs<br/>cargo test + release build"]
    
    LintGates --> MergeDecision{{"Merge Decision"}}
    DocsGates --> MergeDecision
    TestBuild --> MergeDecision
    
    PR --> WorkflowSanity["workflow-sanity.yml<br/>actionlint + tab check"]
    PR --> DockerSmoke["pub-docker-img.yml<br/>Docker smoke test"]
    PR --> SecurityAudit["sec-audit.yml<br/>rustsec + cargo deny"]
    
    WorkflowSanity -.optional.-> MergeDecision
    DockerSmoke -.optional.-> MergeDecision
    SecurityAudit -.optional.-> MergeDecision
    
    MergeDecision --> MainPush["Push to main"]
    MainPush --> DockerPublish["pub-docker-img.yml<br/>Publish images"]
    MainPush --> LabelPolicy["pr-label-policy-check.yml<br/>Policy validation"]
    
    TagPush["Tag push v*"] --> Release["pub-release.yml<br/>Build + publish artifacts"]
    
    Schedule["Daily/Weekly Schedule"] --> Stale["pr-check-stale.yml<br/>Stale PR lifecycle"]
    Schedule --> Hygiene["pr-check-status.yml<br/>Rebase nudges"]
    Schedule --> CodeQL["sec-codeql.yml<br/>Static analysis"]

Sources: docs/ci-map.md:11-38, docs/ci-map.md:66-79

Merge-Blocking vs Optional Checks

Merge-Blocking

Workflow	File	Purpose	Trigger
CI Required Gate	`.github/workflows/ci-run.yml`	Rust quality (fmt, clippy, test, build) + docs quality (incremental markdown + link checks)	Push/PR to `main`
Workflow Sanity	`.github/workflows/workflow-sanity.yml`	Workflow lint (actionlint, tab check)	PR/push when `.github/workflows/**` changes
PR Intake Checks	`.github/workflows/pr-intake-checks.yml`	Template completeness, tabs, trailing whitespace, conflict markers	`pull_request_target` lifecycle

Sources: docs/ci-map.md:11-22

Optional but Important

Workflow	File	Purpose	Trigger
Docker	`.github/workflows/pub-docker-img.yml`	PR smoke check + publish on main/tags	PR + push to `main` (build paths) + tag `v*`
Security Audit	`.github/workflows/sec-audit.yml`	`rustsec/audit-check` + `cargo deny`	Push/PR to `main` + weekly schedule
CodeQL Analysis	`.github/workflows/sec-codeql.yml`	Static security analysis	Weekly schedule + manual
Release	`.github/workflows/pub-release.yml`	Build release artifacts + GitHub releases	Tag push `v*`

Sources: docs/ci-map.md:26-33

CI Quality Gates

The CI Required Gate runs three parallel validation tracks:

flowchart LR
    CIRun["ci-run.yml<br/>CI Required Gate"]
    
    CIRun --> LintTrack["Lint Track"]
    CIRun --> DocsTrack["Docs Track"]
    CIRun --> BuildTrack["Build Track"]
    
    LintTrack --> FmtCheck["cargo fmt --check"]
    LintTrack --> ClippyCheck["cargo clippy --locked<br/>-D clippy::correctness"]
    LintTrack --> StrictDelta["rust_strict_delta_gate.sh<br/>Incremental strict lint"]
    
    DocsTrack --> MarkdownLint["docs_quality_gate.sh<br/>Incremental markdownlint"]
    DocsTrack --> LinkCheck["collect_changed_links.py<br/>+ lychee (added links)"]
    
    BuildTrack --> CargoTest["cargo test --locked"]
    BuildTrack --> ReleaseBuild["cargo build --release<br/>smoke check"]
    
    FmtCheck --> Pass{{"All Green?"}}
    ClippyCheck --> Pass
    StrictDelta --> Pass
    MarkdownLint --> Pass
    LinkCheck --> Pass
    CargoTest --> Pass
    ReleaseBuild --> Pass

Key Implementation Details:

Incremental Strict Lint: ./scripts/ci/rust_strict_delta_gate.sh runs strict clippy (-D warnings) only on changed Rust lines, comparing against BASE_SHA to minimize noise while enforcing strict quality on new/modified code.
Incremental Docs Quality: ./scripts/ci/docs_quality_gate.sh blocks only markdown issues on changed lines, reporting baseline issues separately.
Incremental Link Check: ./scripts/ci/collect_changed_links.py extracts links added on changed lines, then lychee validates only those links to avoid blocking on pre-existing broken links.
Workflow Owner Approval: PRs changing .github/workflows/** require approval from WORKFLOW_OWNER_LOGINS (repository variable fallback: theonlyhennygod,willsarg).

Sources: docs/ci-map.md:13-16, docs/ci-map.md:93-100

Testing Strategy

ZeroClaw uses a three-tier testing approach: unit tests for component isolation, integration tests for subsystem interaction, and E2E tests for agent orchestration workflows.

Test Organization Structure

graph TB
    subgraph "Unit Tests"
        ProviderTests["Provider trait implementations<br/>src/provider/*/mod.rs"]
        ChannelTests["Channel trait implementations<br/>src/channels/*/mod.rs"]
        ToolTests["Tool execution logic<br/>src/tools/*/mod.rs"]
        MemoryTests["Memory backend logic<br/>src/memory/*/mod.rs"]
    end
    
    subgraph "Integration Tests"
        AgentLoop["Agent turn cycle<br/>tests/agent_loop_integration.rs"]
        ConfigLoad["Config loading + validation<br/>tests/config_integration.rs"]
        SecurityPolicy["Security enforcement<br/>tests/security_integration.rs"]
    end
    
    subgraph "E2E Tests"
        ChannelFlow["Channel message processing<br/>tests/e2e_channel_flow.rs"]
        GatewayFlow["Gateway webhook handling<br/>tests/e2e_gateway_flow.rs"]
        DelegationFlow["Sub-agent orchestration<br/>tests/e2e_delegation.rs"]
    end
    
    ProviderTests -.uses.-> AgentLoop
    ChannelTests -.uses.-> ChannelFlow
    ToolTests -.uses.-> AgentLoop
    MemoryTests -.uses.-> AgentLoop
    
    AgentLoop -.uses.-> ChannelFlow
    SecurityPolicy -.uses.-> ChannelFlow

Local Testing Commands

The ./dev/ci.sh script provides unified access to CI validation commands:

Command	Purpose	Maps to CI Job
`./dev/ci.sh fmt`	Run `cargo fmt --all -- --check`	`ci-run.yml` fmt job
`./dev/ci.sh lint`	Run `cargo clippy --locked --all-targets`	`ci-run.yml` clippy job
`./dev/ci.sh lint-strict`	Run full strict clippy audit (`-D warnings`)	Baseline audit (manual)
`./dev/ci.sh lint-delta`	Run strict clippy on changed lines only	`ci-run.yml` lint-strict-delta job
`./dev/ci.sh test`	Run `cargo test --locked`	`ci-run.yml` test job
`./dev/ci.sh build`	Run `cargo build --release`	`ci-run.yml` release-build job
`./dev/ci.sh all`	Run all quality gates sequentially	Full CI simulation

Pre-Push Hook Integration:

The .githooks/pre-push hook calls ./scripts/ci/rust_quality_gate.sh and ./scripts/ci/rust_strict_delta_gate.sh to run incremental quality checks before pushing to remote, catching issues before CI execution.

Sources: docs/ci-map.md:93-100

Test Execution Isolation

Tests requiring external services (LLM APIs, messaging platforms) use mock providers or test doubles to maintain deterministic CI execution without real API keys or network dependencies.

Common Test Patterns:

Provider Mocking: Create test providers that return canned responses for predictable agent behavior validation.
Memory Isolation: Use in-memory SQLite (:memory:) or temporary files for storage-dependent tests.
Tool Sandboxing: Execute shell tools in temporary directories with workspace_only enforcement to prevent filesystem pollution.
Channel Simulation: Use CLIChannel or custom test channels that don't require external messaging service connections.

Code Ownership Structure

ZeroClaw uses CODEOWNERS to enforce review requirements for high-risk surfaces and subsystem-specific expertise. The ownership model follows a last-match-wins pattern where more specific rules override general ownership.

CODEOWNERS Hierarchy

graph TB
    Default["Default Owner<br/>@theonlyhennygod<br/>All files (*)"]
    
    Default --> HighRisk["High-Risk Surfaces"]
    Default --> CI["CI/CD Infrastructure"]
    Default --> Docs["Documentation + Governance"]
    
    HighRisk --> Security["Security<br/>/src/security/**<br/>@willsarg"]
    HighRisk --> Runtime["Runtime<br/>/src/runtime/**<br/>@theonlyhennygod"]
    HighRisk --> Memory["Memory<br/>/src/memory/**<br/>@theonlyhennygod @chumyin"]
    
    CI --> Workflows["Workflows<br/>/.github/workflows/**<br/>@theonlyhennygod @willsarg"]
    CI --> CodeQL["CodeQL<br/>/.github/codeql/**<br/>@willsarg"]
    CI --> Dependabot["Dependabot<br/>/.github/dependabot.yml<br/>@willsarg"]
    
    Docs --> DocFiles["Documentation<br/>/docs/**<br/>@chumyin"]
    Docs --> AgentGuide["Agent Guides<br/>AGENTS.md, CLAUDE.md<br/>@chumyin"]
    Docs --> Contributing["Contribution Docs<br/>CONTRIBUTING.md<br/>@chumyin"]
    
    Docs --> SecurityOverride["Security Override<br/>SECURITY.md<br/>@willsarg (last-match)"]
    Docs --> ActionsPolicy["Actions Policy<br/>docs/actions-source-policy.md<br/>@willsarg (last-match)"]
    Docs --> CIMap["CI Map<br/>docs/ci-map.md<br/>@willsarg (last-match)"]

Sources: .github/CODEOWNERS:1-28

Review Requirements by Path

Path Pattern	Owners	Review Trigger	Rationale
`/src/security/**`	`@willsarg`	Required	Authentication, authorization, sandboxing enforcement
`/src/runtime/**`	`@theonlyhennygod`	Required	Process execution, Docker isolation
`/src/memory/**`	`@theonlyhennygod @chumyin`	Required	Data persistence, query logic, hybrid search
`/.github/workflows/**`	`@theonlyhennygod @willsarg`	Required + owner approval	CI/CD pipeline, merge gates, automation
`/Cargo.toml`, `/Cargo.lock`	`@theonlyhennygod`	Required	Dependency management, supply chain
`/docs/**`	`@chumyin`	Required	Documentation quality, technical writing
`SECURITY.md`	`@willsarg`	Required (override)	Security disclosure policy

Workflow Owner Approval Logic:

PRs modifying .github/workflows/** must pass an additional check in the CI Required Gate that verifies at least one approving review from a login in the WORKFLOW_OWNER_LOGINS repository variable (defaults to theonlyhennygod,willsarg). This prevents unauthorized modification of merge-blocking checks and prevents accidental CI pipeline breakage.

Sources: docs/ci-map.md:15, .github/CODEOWNERS:1-28

Local Development Scripts

ZeroClaw provides several scripts to replicate CI validation locally:

Primary Scripts

Script	Purpose	Usage
`./dev/ci.sh`	Unified CI command runner	`./dev/ci.sh <command>` (fmt, lint, test, build, all)
`./scripts/ci/rust_quality_gate.sh`	Standard quality checks	Called by `./dev/ci.sh`, `.githooks/pre-push`
`./scripts/ci/rust_strict_delta_gate.sh`	Incremental strict lint	`./dev/ci.sh lint-delta` or pre-push hook
`./scripts/ci/docs_quality_gate.sh`	Incremental markdown lint	Called by `ci-run.yml` docs-quality job
`./scripts/ci/collect_changed_links.py`	Extract added links	Called by `ci-run.yml` docs-quality job + lychee

Pre-Push Hook Setup

Install git hooks to run quality checks before pushing:

git config core.hooksPath .githooks

The .githooks/pre-push hook executes:

./scripts/ci/rust_quality_gate.sh (fmt, clippy baseline)
./scripts/ci/rust_strict_delta_gate.sh (strict lint on changed lines)

Sources: docs/ci-map.md:95-96

Incremental Quality Gate Workflow

The strict delta lint gate runs incremental validation to minimize friction:

sequenceDiagram
    participant Dev as Developer
    participant Hook as .githooks/pre-push
    participant DeltaGate as rust_strict_delta_gate.sh
    participant Git as Git Commands
    
    Dev->>Hook: git push
    Hook->>DeltaGate: Execute with BASE_SHA
    DeltaGate->>Git: git diff --name-only BASE_SHA HEAD
    Git-->>DeltaGate: Changed Rust files
    
    loop For each changed file
        DeltaGate->>Git: git diff -U0 BASE_SHA HEAD <file>
        Git-->>DeltaGate: Changed line ranges
        DeltaGate->>DeltaGate: cargo clippy --message-format=json
        DeltaGate->>DeltaGate: Filter warnings by changed lines
    end
    
    alt All changed lines pass strict lint
        DeltaGate-->>Hook: Exit 0
        Hook-->>Dev: Push proceeds
    else Strict lint failure on changed lines
        DeltaGate-->>Hook: Exit 1 + error report
        Hook-->>Dev: Push blocked with fix commands
    end

Key Benefit: Developers can incrementally improve code quality on lines they touch without requiring full codebase remediation. The baseline strict lint audit (./dev/ci.sh lint-strict) runs separately to track overall technical debt.

Sources: docs/ci-map.md:96-98

PR Size Policy and Batching

ZeroClaw enforces size tiers to keep PRs reviewable and merge velocity predictable:

Size Label	Changed Lines	Review Expectation
`size: XS`	≤ 80	Fast triage, quick approval
`size: S`	≤ 250	Standard review
`size: M`	≤ 500	Focused review with scope validation
`size: L`	≤ 1000	Requires explicit justification + tighter test evidence
`size: XL`	> 1000	Requires split into stacked PRs unless strongly justified

Automation Behavior:

The PR Labeler workflow applies size:* labels from effective changed lines, normalizing for docs-only or lockfile-heavy PRs to avoid size inflation.
Docs-only PRs automatically skip heavy Rust validation jobs in ci-run.yml via path filters.

Recommended Strategy:

Target size: XS/S/M by default. If a large feature is unavoidable, split into stacked PRs with explicit Depends on #... linkage so review order is deterministic.

Sources: docs/pr-workflow.md:162-182

Risk-Based Review Depth

ZeroClaw uses automated risk classification to route PRs to appropriate review depth:

Risk Classification Triggers

Risk Level	Trigger Paths	Review Requirements
`risk: high`	`src/security/`, `src/runtime/`, `src/gateway/`, `src/tools/`, `.github/workflows/**`	Threat statement, mitigation notes, rollback steps, failure-mode scenario
`risk: medium`	Provider implementations, channel implementations, memory backends	Security impact fields, rollback plan
`risk: low`	Documentation, tests, configuration examples	Standard template completion

Manual Override:

Maintainers can apply risk: manual to preserve a human risk judgment when automation lacks context or when nuanced security considerations require explicit override.

Sources: docs/pr-workflow.md:51-56, docs/pr-workflow.md:330-333

Failure Recovery Protocol

When a merged PR causes regressions:

Revert immediately on main to restore service quality.
Open follow-up issue with root-cause analysis and regression test plan.
Re-introduce fix only after regression tests prove the issue is resolved.

Philosophy: Prefer fast restore of service quality over delayed perfect fixes. The revert-first approach minimizes blast radius and allows thorough investigation without production pressure.

Sources: docs/pr-workflow.md:270-278

Maintainer Merge Checklist

Before merging, maintainers verify:

Scope is focused and understandable
CI Required Gate is green
Docs-quality checks are green when docs changed
Security impact fields are complete for risky paths
Privacy/data-hygiene fields are complete and evidence is redacted/anonymized
Agent workflow notes are sufficient for reproducibility (if automation was used)
Rollback plan is explicit and concrete
Commit title follows Conventional Commits format

Sources: docs/pr-workflow.md:281-291

Agent-Assisted Contribution Guidelines

AI-assisted PRs are welcome. Review can also be agent-assisted.

Required for AI-Heavy PRs

Clear PR summary with scope boundary (what changed / what did not)
Explicit test/validation evidence (not just "CI will check")
Security impact and rollback notes for risky changes

Review Emphasis for AI-Heavy PRs

Reviewers focus on:

Contract compatibility (does the change respect trait boundaries?)
Security boundaries (are path checks, allowlists, and sandboxing preserved?)
Error handling and fallback behavior
Performance and memory regressions

Sources: docs/pr-workflow.md:185-208

12 Development Guide

Development Guide

Purpose and Scope

Development Workflow Overview

PR Lifecycle Stages

Key Workflow Characteristics

CI/CD Architecture

CI Workflow Integration Map

Merge-Blocking vs Optional Checks

Merge-Blocking

Optional but Important

CI Quality Gates

Testing Strategy

Test Organization Structure

Local Testing Commands

Test Execution Isolation

Code Ownership Structure

CODEOWNERS Hierarchy

Review Requirements by Path

Local Development Scripts

Primary Scripts

Pre-Push Hook Setup

Incremental Quality Gate Workflow

PR Size Policy and Batching

Risk-Based Review Depth

Risk Classification Triggers

Failure Recovery Protocol

Maintainer Merge Checklist

Agent-Assisted Contribution Guidelines

Required for AI-Heavy PRs

Recommended

Review Emphasis for AI-Heavy PRs

Related Documentation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!