Skip to content

Latest commit

 

History

History
434 lines (325 loc) · 19.6 KB

File metadata and controls

434 lines (325 loc) · 19.6 KB

Contributing to OmniRoute

Thank you for your interest in contributing! This guide covers everything you need to get started.

For the official per-change workflow, start with the Contribution Golden Path. It maps provider, routing, UI/UX, i18n, CLI, database, and build/deploy changes to their contracts, focused tests, CI coverage, and reconciliation steps.


Development Setup

Prerequisites

  • Node.js >=22.22.3 <23, or >=24.0.0 <27 (recommended: 24 LTS)
  • npm 10+

npm v11+ users (Node 24+): After npm install, verify native modules were installed: node -e "require('better-sqlite3')". If it fails with MODULE_NOT_FOUND, run npm approve-scripts better-sqlite3 && npm install. See Troubleshooting.

  • Git

Clone & Install

git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
npm install

Environment Variables

# Create your .env from the template
cp .env.example .env

# Generate required secrets
echo "JWT_SECRET=$(openssl rand -base64 48)" >> .env
echo "API_KEY_SECRET=$(openssl rand -hex 32)" >> .env

Key variables for development:

Variable Development Default Description
PORT 20128 Server port
NEXT_PUBLIC_BASE_URL http://localhost:20128 Base URL for frontend
JWT_SECRET (generate above) JWT signing secret
INITIAL_PASSWORD CHANGEME First login password
APP_LOG_LEVEL info Log verbosity level

Dashboard Settings

The dashboard provides UI toggles for features that can also be configured via environment variables:

Setting Location Toggle Description
Settings → Advanced Debug Mode Enable debug request logs (UI)
Settings → General Sidebar Visibility Show/hide sidebar sections

These settings are stored in the database and persist across restarts, overriding env var defaults when set.

Running Locally

# Development mode (hot reload)
npm run dev

# Production build
npm run build    # next build → .build/next/ then assembleStandalone → dist/
npm run start

# Fast backend/API-only compile for contributor changes
npm run build:contributor

# Release build (clean rebuild + HEAD sentinel — required for deploy)
npm run build:release   # rm -rf .build dist && build + writes dist/BUILD_SHA

# Common port configuration
PORT=20128 NEXT_PUBLIC_BASE_URL=http://localhost:20128 npm run dev

The contributor build performs compile-only validation: it does not assemble the standalone distribution or build optional native packaging assets. Use the regular production build when you need to validate the shippable bundle.

Build Output Layout

Directory Contents Tracked
src/ Application source (TypeScript / TSX) Yes
.build/ Intermediates — next build output (gitignored, distDir = .build/next) No
dist/ Shippable bundle — assembled by assembleStandalone (gitignored) No

The build pipeline is a single pass:

npm run build
  └─ next build → .build/next/standalone  (Next.js output)
  └─ assembleStandalone()                 (copies standalone + static + public + native assets)
       └─ output: dist/                   (server.js, .next/static/, public/, node_modules/)

npm run build:release additionally cleans both directories first and writes dist/BUILD_SHA (= git rev-parse --short HEAD) as a deploy integrity sentinel.

npm run build:contributor uses the backend-only build profile. It temporarily stubs dashboard UI files while building, keeps API route handlers, and restores the original files after the build. Use npm run build for changes that affect the dashboard UI or for full release validation; the contributor profile is not a replacement for the release build.

VPS deploy note: the remote image directory /usr/lib/node_modules/omniroute/app/ is unchanged. The deploy skills rsync the contents of dist/ into it. Only the in-repo build output path moved (app/dist/).

Default URLs:

  • Dashboard: http://localhost:20128/dashboard
  • API: http://localhost:20128/v1

Git Workflow

⚠️ NEVER commit directly to main. Always use feature branches.

PR base: target the active release/vX.Y.Z branch (not main). See docs/ops/BRANCHING_MODEL.md for the release-per-branch + tag-at-ship model.

# Branch from the active release tip (example: release/v3.8.49)
git fetch origin
git checkout -b feat/your-feature-name origin/release/v3.8.49
# ... make changes ...
git commit -m "feat: describe your change"
git push -u origin feat/your-feature-name
# Open a Pull Request with base = release/v3.8.49

Branch Naming

Prefix Purpose
feat/ New features
fix/ Bug fixes
refactor/ Code restructuring
docs/ Documentation changes
test/ Test additions/fixes
chore/ Tooling, CI, dependencies

Commit Messages

Follow Conventional Commits:

feat: add circuit breaker for provider calls
fix: resolve JWT secret validation edge case
docs: update SECURITY.md with PII protection
test: add observability unit tests
refactor(db): consolidate rate limit tables

Scopes (v3.8): db, sse, oauth, dashboard, api, cli, docker, ci, mcp, a2a, memory, skills, cloud-agent, guardrails, compression, auto-combo, resilience, providers, executors, translator, domain, authz.


Running Tests

# All tests (unit + vitest + ecosystem + e2e)
npm run test:all

# Single test file (Node.js native test runner — most tests use this)
node --import tsx/esm --test tests/unit/your-file.test.ts

# Only the unit tests impacted by your change (same TIA selector as the CI gate, #8084)
npm run test:scoped            # changes in the last commit (or the working tree)
npm run test:scoped:staged     # staged changes only — pairs well with a pre-commit run
npm run test:scoped:full       # rebuild the import-graph map first (after adding/moving files)
# Exit 1 + "run the full suite" means a hub file (tsconfig, package.json, …) or an
# unmapped source changed — the selector fails safe, it never silently skips.

# Vitest (MCP server, autoCombo, cache)
npm run test:vitest

# E2E tests (requires Playwright)
npm run test:e2e

# Protocol clients E2E (MCP transports, A2A)
npm run test:protocols:e2e

# Ecosystem compatibility tests
npm run test:ecosystem

# Coverage gate: 60% statements/lines/functions/branches
npm run test:coverage
npm run coverage:report

# Lint + format check
npm run lint
npm run check

# Gated real-upstream combo smoke (requires VPS access + real provider credits)
# Hits REAL providers — costs a little. NEVER runs in CI. Skips cleanly without the gate.
# Needs: ssh root@192.168.0.15 access (sources a read-only DB snapshot from the VPS).
RUN_COMBO_LIVE=1 npm run test:combo:live

# Phase-3 VPS live smoke — plain Node ESM scripts, hit the live .15 server directly.
# Requires: ssh root@192.168.0.15 access (combos created/torn down via SSH sqlite).
# Hits REAL providers (small cost). Creates/deletes only __live_test__* combos. NEVER runs in CI.
# REQUIRE_API_KEY=false on .15 so no API key needed, but honors COMBO_LIVE_BASE_URL / COMBO_LIVE_API_KEY if set.
npm run test:combo:live:vps              # 7 HTTP scenarios (priority/round-robin/weighted/cost/fusion/auto + health)
npm run test:combo:live:vps:failover     # adds a real cross-provider failover scenario (8 total)

Coverage notes:

  • npm run test:coverage measures source coverage for the main unit test suite, excludes tests/**, and includes open-sse/**
  • Pull requests must keep the coverage gate at 60%+ statements/lines/functions/branches
  • If a PR changes production code in src/, open-sse/, electron/, or bin/, it must add or update automated tests in the same PR
  • npm run coverage:report prints the detailed file-by-file report from the latest coverage run
  • npm run test:coverage:legacy preserves the older metric for historical comparison
  • See docs/ops/COVERAGE_PLAN.md for the phased coverage improvement roadmap

Pull Request Requirements

Before opening a PR, use the Contribution Golden Path to run the focused loop for what you changed. The full unit suite (4 CI shards), Vitest, the 60%+ coverage gate, and the production build are CI's responsibility — running them locally adds no signal the PR checks will not already give you, and on smaller machines it can saturate the host (#8084):

  • Run the test files that cover your change: node --import tsx/esm --test tests/unit/<file>.test.ts
  • Run npm run lint
  • Include or update automated tests in the same PR whenever production code changes
  • Include the changed or added test files in the PR description when production code changed
  • Check the SonarQube result on the PR when the project secrets are configured in CI

Current test status: 122 unit test files covering:

  • Provider translators and format conversion
  • Rate limiting, circuit breaker, and resilience
  • Semantic cache, idempotency, progress tracking
  • Database operations and schema (21 DB modules)
  • OAuth flows and authentication
  • API endpoint validation (Zod v4)
  • MCP server tools and scope enforcement
  • Memory and Skills systems

Code Style

  • ESLint — Run npm run lint before committing
  • Prettier — Auto-formatted via lint-staged on commit (2 spaces, semicolons, double quotes, 100 char width, es5 trailing commas)
  • TypeScript — All src/ code uses .ts/.tsx; open-sse/ uses .ts/.js; document with TSDoc (@param, @returns, @throws)
  • No eval() — ESLint enforces no-eval, no-implied-eval, no-new-func
  • Zod validation — Use Zod v4 schemas for all API input validation
  • Naming: Files = camelCase/kebab-case, components = PascalCase, constants = UPPER_SNAKE

Error handling / empty catch blocks

Never leave a catch unexplained. Classify it into one of two buckets (operationalizes the hard rule "never silently swallow errors in SSE streams"):

  • Intentional (our own best-effort cleanup/telemetry) — a failure here is expected and harmless; add a one-line rationale comment, no logging (logging on every request is the noise this convention avoids).

    } catch {} // closing an already-closed controller after client disconnect is expected
  • Should log (external/caller-supplied code, or the swallow changes control flow) — keep the catch (never let it break the stream) but emit a contextual console.debug/warn so the failure is discoverable.

    } catch (e) {
      console.debug("[STREAM] onFailure callback error:", e);
    }

See open-sse/utils/stream.ts and open-sse/utils/streamHandler.ts for applied examples.


Project Structure

src/                        # TypeScript (.ts / .tsx)
├── app/                    # Next.js 16 App Router
│   ├── (dashboard)/        # Dashboard pages (23 sections)
│   ├── api/                # API routes (51 directories)
│   └── login/              # Auth pages (.tsx)
├── domain/                 # Policy engine (policyEngine, comboResolver, costRules, etc.)
├── lib/                    # Core business logic (.ts)
│   ├── a2a/                # Agent-to-Agent v0.3 protocol server
│   ├── acp/                # Agent Communication Protocol registry
│   ├── compliance/         # Compliance policy engine
│   ├── db/                 # SQLite domain modules + 130 migrations
│   ├── memory/             # Persistent conversational memory
│   ├── oauth/              # OAuth providers, services, and utilities
│   ├── skills/             # Extensible skill framework
│   ├── usage/              # Usage tracking and cost calculation
│   └── localDb.ts          # Re-export layer only — never add logic here
├── middleware/              # Request middleware (promptInjectionGuard)
├── mitm/                   # MITM proxy (cert, DNS, target routing)
├── shared/
│   ├── components/         # React components (.tsx)
│   ├── constants/          # Provider definitions (329), MCP scopes, 19 routing strategies
│   ├── utils/              # Circuit breaker, sanitizer, auth helpers
│   └── validation/         # Zod v4 schemas
└── sse/                    # SSE proxy pipeline

open-sse/                   # @omniroute/open-sse workspace
├── executors/              # 89 executor implementation modules
├── handlers/               # 11 request handlers (chat, responses, embeddings, images, etc.)
├── mcp-server/             # MCP server (110 unique tools, 3 transports, 33 scopes)
├── services/               # 178 top-level services (combo, autoCombo, rateLimitManager, etc.)
├── translator/             # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Ollama)
├── transformer/            # Responses API transformer
└── utils/                  # 22 utility modules (stream, TLS, proxy, logging)

electron/                   # Electron desktop app (cross-platform)

tests/
├── unit/                   # Node.js test runner (1,574 test files)
├── integration/            # Integration tests
├── e2e/                    # Playwright tests
├── security/               # Security tests
├── translator/             # Translator-specific tests
└── load/                   # Load tests

docs/
├── adr/                     # Architecture Decision Records
├── architecture/            # System architecture & resilience
├── comparison/              # OmniRoute vs alternatives
├── compression/             # Compression guides & rules
├── dev/                     # Development guides
├── diagrams/                # Architecture diagrams
├── frameworks/              # MCP, A2A, OpenCode, Memory, Skills
├── guides/                  # User guide, Docker, setup, troubleshooting
├── i18n/                    # Internationalized README translations
├── marketing/               # Marketing materials
├── ops/                     # Deployment, proxy, coverage, releases
├── providers/               # Provider-specific docs
├── reference/               # API reference, env vars, CLI tools, free tiers
├── releases/                # Release notes
├── routing/                 # Auto-combo engine, reasoning replay
├── screenshots/             # Dashboard screenshots
├── security/                # Guardrails, compliance, stealth, tokens
└── specs/                   # Design specs

Adding a New Provider

Step 1: Register Provider Constants

Add to src/shared/constants/providers.ts — Zod-validated at module load.

Step 2: Add Executor (if custom logic needed)

Create executor in open-sse/executors/your-provider.ts extending the base executor.

Step 3: Add Translator (if non-OpenAI format)

Create request/response translators in open-sse/translator/.

Step 4: Add OAuth Config (if OAuth-based)

Add OAuth credentials in src/lib/oauth/constants/oauth.ts and service in src/lib/oauth/services/.

If the upstream provider distributes a public OAuth client_id/secret or Firebase Web API key inside its public CLI / browser bundle, do not embed it as a string literal. Use resolvePublicCred() from open-sse/utils/publicCreds.ts and add a masked byte entry to EMBEDDED_DEFAULTS. The full mandatory workflow is documented in docs/security/PUBLIC_CREDS.md.

Inside handlers/executors, error messages reaching the client must go through buildErrorBody() / sanitizeErrorMessage() from open-sse/utils/error.ts — never put raw err.stack or err.message in a Response body. See docs/security/ERROR_SANITIZATION.md.

Step 5: Register Models

Add model definitions in open-sse/config/providerRegistry.ts.

Step 6: Add Tests

Write unit tests in tests/unit/ covering at minimum:

  • Provider registration
  • Request/response translation
  • Error handling

Pull Request Checklist

  • Tests pass (npm test)
  • Linting passes (npm run lint)
  • Build succeeds (npm run build)
  • TypeScript types added for new public functions and interfaces
  • No hardcoded secrets or fallback values
  • Public upstream credentials embedded via resolvePublicCred() (see docs/security/PUBLIC_CREDS.md), never as literals
  • Error responses route through buildErrorBody() / sanitizeErrorMessage() — no raw stack traces in response bodies (see docs/security/ERROR_SANITIZATION.md)
  • Shell commands (exec / spawn) pass runtime values via env, not via string interpolation
  • All inputs validated with Zod schemas
  • Changelog fragment added under changelog.d/{features|fixes|maintenance}/<PR>-<slug>.md for user-facing changes (see changelog.d/README.md) — do not edit CHANGELOG.md directly; fragments are aggregated at release time and never conflict between PRs
  • Documentation updated (if applicable)
  • No new CodeQL / Secret-Scanning alerts opened, or each one dismissed with technical justification referencing the relevant docs/security/ doc
  • Routes that spawn child processes (/api/mcp/, /api/cli-tools/runtime/) classified as isLocalOnlyPath() in src/server/authz/routeGuard.ts — see Hard Rule #15
  • No Co-Authored-By trailers in commit messages — commits must appear solely under the repository owner's Git identity (Hard Rule #16)

Releasing

Releases are managed via the /generate-release workflow. When a new GitHub Release is created, the package is automatically published to npm via GitHub Actions.

For VPS deploys, use npm run build:release (not npm run build) — it performs a clean rebuild, assembles the bundle into dist/, and writes the dist/BUILD_SHA sentinel. Then use the /deploy-vps-*-cc skills which rsync dist/ to the remote app/ directory.


Getting Help