Skip to content
185 changes: 185 additions & 0 deletions OMOA-FEATURES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# OMOA - Model Management System for Oh My Open Agent

## Overview

OMOA is a policy-based model routing and config management system. It replaces manual JSON editing with two complementary modes:

1. **Policy Mode** -- Declare desired model rankings per agent/category. OMOA auto-resolves the best model based on provider availability, compatibility rules, and cross-provider fallback constraints.
2. **Manual Mode** -- Direct per-field editing of any config value via a schema-driven TUI that auto-generates prompts from the project's Zod schemas.

Both modes are accessible from the same TUI and CLI.

## Commands

```
omoa Interactive TUI (main entry)
omoa status Show providers, agents, categories, validation
omoa build Build config from rankings + provider state
omoa build --dry-run Preview changes without writing
omoa build --yes Skip confirmation
omoa provider list Show provider status and usage counts
omoa provider enable <n> Enable a provider
omoa provider disable <n> Disable a provider
```

## Architecture

### Three Layers of State

| File | Purpose |
|------|---------|
| `oh-my-openagent.json` | Runtime config (written by `omoa build` or manual edits) |
| `omoa-state.json` | OMOA state: provider enable/disable, banned models, rules |
| `omoa-rankings.json` | Per-agent/category model preference lists |

### How It Works

```
omoa-state.json (providers, rules)
+
omoa-rankings.json (model preferences)
|
v
[Resolver Engine]
- Filter by provider availability
- Filter by banned/deprecated models
- Filter by free-only rules
- Pick first available as primary
- Pick first cross-provider model as fallback
- Check avoid_fallback_from rules
|
v
[Builder]
- Compare resolved vs current config
- Generate diff (AgentChange[])
- Create backup
- Write only model/fallback fields
- Preserve all other fields untouched
|
v
oh-my-openagent.json (runtime config)
```

### Key Design Rules

- **Idempotent build**: Running `omoa build` with no changes produces no output.
- **Additive**: Build only changes `model` and `fallback_models` fields. Temperature, thinking, permissions, etc. are never touched.
- **Cross-provider fallback**: `primary.provider != fallback.provider` is enforced.
- **Managed vs manual**: Agents with rankings are "OMOA-managed" (built automatically). Agents without rankings are "manual" (edited directly). Both coexist.
- **Schema-driven editor**: All non-model fields are edited via Zod shape introspection. New fields in the schema automatically appear in the editor.

## File Structure

```
src/cli/omoa/
index.ts CLI entry point + command implementations
state/
omoa-state-schema.ts Zod schema for omoa-state.json
omoa-rankings-schema.ts Zod schema for omoa-rankings.json
state-manager.ts Read/write/mutate omoa-state.json
rankings-manager.ts Read/write/mutate omoa-rankings.json
engine/
resolver.ts resolveBestModel(rankings, state) -> {primary, fallback, reasons}
validator.ts 10-rule validation system
builder.ts Orchestrate: resolve -> diff -> backup -> write
models/
model-cache.ts Zod-validated model cache reader (provider-models.json, models.json)
tui/
main-menu.ts Interactive TUI main menu
provider-screen.ts Provider toggle screen
ranking-screen.ts Rankings view/edit screen
assign-screen.ts Multi-target model assignment screen
shared.ts Shared helpers (loadRuntimeConfig, countProviderUsage)
schema-editor/
agent-editor.ts Agent field editor (schema-driven)
category-editor.ts Category field editor
root-editor.ts Root config field editor
field-renderer.ts Zod type -> @clack/prompts mapping
```

## Validation Rules

1. Config file exists and is valid JSON
2. All primary models belong to enabled providers
3. All fallback models belong to enabled providers
4. Primary provider != fallback provider
5. Opencode provider: only `*-free` models allowed (when free_only is set)
6. No banned models active
7. Deprecated models trigger warnings
8. Missing fallbacks are informational warnings
9. Same-provider fallbacks trigger warnings
10. Doctor can be run for deeper checks

## Dual-Mode Design

### Policy Mode (OMOA-managed)
- Agent/category has a ranking in `omoa-rankings.json`
- `omoa build` resolves model automatically
- Model field shows "[OMOA]" badge in status
- Manual model edits warn: "will be overwritten on next build"

### Manual Mode
- Agent/category has NO ranking entry
- User edits model directly via "Edit Config" TUI
- Model field shows "[manual]" badge in status
- `omoa build` skips this agent/category entirely

Both modes share the same schema-driven editor for non-model fields (temperature, thinking, permissions, etc.).

## Schema-Driven Editor

The field renderer maps Zod types to `@clack/prompts` UIs:

| Zod Type | TUI Prompt |
|----------|-----------|
| `z.string()` | Text input |
| `z.boolean()` | Select: true/false/clear |
| `z.number()` | Text input with number validation |
| `z.enum([...])` | Select from enum values |
| `z.object({...})` | JSON text input |
| `z.record(...)` | JSON text input |
| `z.array(...)` | Comma-separated text input |

This means new fields added to `AgentOverrideConfigSchema` or `CategoryConfigSchema` automatically appear in the editor without code changes.

## Features Discussed

### Phase 1 (Implemented)
- [x] OMOA state file with provider rules, banned/deprecated models
- [x] Rankings file with per-agent/category model preference lists
- [x] Core resolver engine (provider availability, cross-provider fallback, compatibility rules)
- [x] Build command with dry-run support
- [x] Status command (providers, agents, categories, validation)
- [x] Provider enable/disable with usage counts
- [x] Multi-target model assignment
- [x] Schema-driven field editor for all agent/category/root fields
- [x] Manual model editing for non-OMOA-managed agents
- [x] Backup before every write operation
- [x] 10-rule validation system
- [x] Interactive TUI with all screens
- [x] CLI subcommands for scripting

### Phase 2 (Planned)
- [ ] `omoa preset balanced` / `omoa preset emergency-free`
- [ ] `omoa restore` (list and restore backups)
- [ ] Ranking reordering in TUI (move up/down, add/remove models)
- [ ] Deep Zod shape introspection (auto-detect field types from schema instead of hardcoded field lists)
- [ ] `omoa doctor` integration (wrap existing doctor command)
- [ ] `omoa rankings edit <agent>` CLI subcommand
- [ ] `omoa assign <model> --to <agent1,agent2>` CLI flags
- [ ] Category ranking management in TUI
- [ ] Provider compatibility matrix (avoid_fallback_from editing in TUI)
- [ ] FallbackModelObject support (model + variant + thinking per fallback entry)

### Phase 3 (Planned)
- [ ] `omoa init` wizard (bootstrap omoa-state.json + omoa-rankings.json from current config)
- [ ] Import rankings from existing config (detect current models -> generate rankings)
- [ ] Live provider availability from API/cache polling
- [ ] Model performance hints (cost tier, speed tier)
- [ ] Config diff viewer (show what changed between builds)
- [ ] Undo/redo stack for config changes

## Dependencies

- Uses existing project dependencies only: `@clack/prompts`, `picocolors`, `zod`, `commander`, `jsonc-parser`
- No new dependencies added
46 changes: 46 additions & 0 deletions src/cli/cli-program.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { doctor } from "./doctor"
import { refreshModelCapabilities } from "./refresh-model-capabilities"
import { createMcpOAuthCommand } from "./mcp-oauth"
import { boulder } from "./boulder"
import { omoaInteractive, omoaStatus, omoaBuild, omoaProviderList, omoaProviderSet, omoaConfig } from "./omoa"
import type { InstallArgs } from "./types"
import type { RunOptions } from "./run"
import type { GetLocalVersionOptions } from "./get-local-version/types"
Expand Down Expand Up @@ -198,6 +199,51 @@ program
process.exit(exitCode)
})

program
.command("omoa")
.description("OMOA model management - policy-based model routing for agents and categories")
.addHelpText("after", `
Subcommands:
omoa Interactive TUI
omoa status Show provider/agent/category overview
omoa build Build config from rankings + provider state
omoa build --dry-run Preview changes without writing
omoa build --yes Skip confirmation
omoa provider list Show provider status
omoa provider enable <name> Enable a provider
omoa provider disable <name> Disable a provider
omoa config Interactive config editor

Examples:
$ bunx oh-my-opencode omoa
$ bunx oh-my-opencode omoa status
$ bunx oh-my-opencode omoa build --dry-run
$ bunx oh-my-opencode omoa provider disable openai
`)
.argument("[subcommand]", "Subcommand: status, build, provider")
.argument("[args...]", "Subcommand arguments")
.option("--dry-run", "Preview changes without writing (build)")
.option("--yes", "Skip confirmation (build)")
.action(async (subcommand, args, options) => {
let exitCode: number
if (subcommand === "status") {
exitCode = omoaStatus()
} else if (subcommand === "build") {
exitCode = omoaBuild(options.dryRun ?? false, options.yes ?? false)
} else if (subcommand === "provider" && args?.[0] === "list") {
exitCode = omoaProviderList()
} else if (subcommand === "provider" && args?.[0] === "enable" && args?.[1]) {
exitCode = omoaProviderSet(args[1], true)
} else if (subcommand === "provider" && args?.[0] === "disable" && args?.[1]) {
exitCode = omoaProviderSet(args[1], false)
} else if (subcommand === "config") {
exitCode = await omoaConfig()
} else {
exitCode = await omoaInteractive()
}
process.exit(exitCode)
})

program
.command("version")
.description("Show version information")
Expand Down
109 changes: 109 additions & 0 deletions src/cli/config/__tests__/config-editor.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
import { describe, test, expect } from "bun:test"
import { validateConfig, checkFallbackWarnings, countWarnings } from "../validation"
import type { ConfigEditorState } from "../types"

function makeState(config: Record<string, unknown>): ConfigEditorState {
return {
config: config as ConfigEditorState["config"],
modified: false,
configPath: "/tmp/test-oh-my-opencode.json",
}
}

describe("config editor validation", () => {
test("empty config produces no validation warnings", () => {
const state = makeState({
agents: {},
categories: {},
})
const warnings = validateConfig(state)
expect(warnings.length).toBe(0)
})

test("config with model on all agents produces no warnings", () => {
const state = makeState({
agents: {
build: { model: "openai/gpt-5.5" },
plan: { model: "openai/gpt-5.5" },
sisyphus: { model: "anthropic/claude-opus-4-7" },
},
categories: {},
})
const warnings = validateConfig(state)
expect(warnings.length).toBe(0)
})

test("agent with missing model produces warning", () => {
const state = makeState({
agents: {
sisyphus: {},
},
categories: {},
})
const warnings = validateConfig(state)
const noModel = warnings.find((w) => w.agent === "sisyphus" && w.type === "missing-model")
expect(noModel).toBeDefined()
})

test("agent with model but no fallback produces fallback warning", () => {
const state = makeState({
agents: {
sisyphus: { model: "anthropic/claude-opus-4-7" },
},
categories: {},
})
const warnings = checkFallbackWarnings(state)
const noFallback = warnings.find((w) => w.agent === "sisyphus" && w.type === "missing-fallback")
expect(noFallback).toBeDefined()
})

test("agent with model and fallback produces no fallback warning", () => {
const state = makeState({
agents: {
sisyphus: {
model: "anthropic/claude-opus-4-7",
fallback_models: ["openai/gpt-5.5"],
},
},
categories: {},
})
const warnings = checkFallbackWarnings(state)
const noFallback = warnings.find((w) => w.agent === "sisyphus" && w.type === "missing-fallback")
expect(noFallback).toBeUndefined()
})
})

describe("config editor types", () => {
test("AGENT_NAMES includes expected agents", async () => {
const { AGENT_NAMES } = await import("../types")
expect(AGENT_NAMES).toContain("sisyphus")
expect(AGENT_NAMES).toContain("oracle")
expect(AGENT_NAMES).toContain("build")
expect(AGENT_NAMES).toContain("plan")
expect(AGENT_NAMES.length).toBeGreaterThan(5)
})

test("BUILTIN_CATEGORIES includes expected categories", async () => {
const { BUILTIN_CATEGORIES } = await import("../types")
expect(BUILTIN_CATEGORIES).toContain("visual-engineering")
expect(BUILTIN_CATEGORIES).toContain("deep")
expect(BUILTIN_CATEGORIES).toContain("ultrabrain")
})
})

describe("config editor models", () => {
test("getModelsByProvider returns record without errors", async () => {
const { getModelsByProvider } = await import("../models")
const result = getModelsByProvider()
// Should always return a valid object (possibly empty if no cache files)
expect(typeof result).toBe("object")
expect(Array.isArray(result)).toBe(false)
// Can't assert specific providers since cache file may not exist
})

test("getAllCachedModels returns sorted list", async () => {
const { getAllCachedModels } = await import("../models")
const result = getAllCachedModels()
expect(Array.isArray(result)).toBe(true)
})
})
Loading
Loading