Skip to content

refactor(pi-ai): replace model-ID pattern matching with model capability metadata #2546

@jeremymcs

Description

@jeremymcs

Summary

The codebase has a systemic pattern of checking capability support by parsing model ID strings (model.id.includes(\"gpt-5.4\"), model.id.startsWith(\"claude-\")) or hardcoding provider names (model.provider === \"xai\") at call sites. This violates the provider-agnostic principle in VISION.md — any new model or provider that doesn't match a hardcoded pattern breaks silently.

The correct fix: declare capabilities in Model<TApi> metadata so capability checks read from the model object, not from ID strings.


Findings (full audit)

HIGH — breaks provider-agnostic principle

File Issue
packages/pi-ai/src/models.ts:74-84 supportsXhigh() hardcodes gpt-5.2/5.3/5.4 and opus-4-6/4.6 ID patterns
src/cli.ts:389-393 Default model fallback hardcodes OpenAI-first then Anthropic priority order
src/web/onboarding-service.ts:336-379 Provider API endpoints and version headers hardcoded in core onboarding service
src/resources/extensions/gsd/service-tier.ts:34-38 supportsServiceTier() gates on gpt-5.4 prefix — no extension point for other providers

MEDIUM — cross-provider leakage in provider files

File Issue
packages/pi-ai/src/providers/google-shared.ts:70 requiresToolCallId() checks for claude- and gpt-oss- prefixes inside a Google provider file
packages/pi-ai/src/providers/google-gemini-cli.ts:207,919 Google provider explicitly detects Claude models by name
packages/pi-ai/src/providers/amazon-bedrock.ts:229,687 Bedrock provider hardcodes anthropic.claude / anthropic/claude pattern checks
packages/pi-ai/src/providers/openai-completions.ts:393 OpenAI handler special-cases Anthropic models routed via OpenRouter
src/resources/extensions/gsd/token-counter.ts Hardcoded chars-per-token ratios per provider; falls back to gpt-4o tiktoken as universal tokenizer
src/resources/extensions/gsd/model-router.ts Static model-to-tier and model-to-cost tables — adding new models requires code changes
src/resources/extensions/gsd/prompt-cache-optimizer.ts:191-198 Cache efficiency multipliers hardcoded per provider
src/resources/extensions/search-the-web/native-search.ts Deeply coupled to Anthropic's web_search_20250305 tool

LOW / Cosmetic

File Issue
src/resources/extensions/gsd/export-html.ts Shortens claude- prefixes in display but not other providers
src/resources/extensions/gsd/preferences-models.ts Default model profiles are all Claude (overridable, but still)

Root cause

The codebase conflates model name patterns with provider capabilities. The Model<TApi> type already has compat fields for OpenAI-compatible APIs — the pattern exists. It needs to be extended with a general capabilities field so any model or provider can declare what it supports without requiring changes to call-site logic.


Proposed fix

1. Add ModelCapabilities to types.ts

export interface ModelCapabilities {
  /** Whether the model supports xhigh thinking level. */
  supportsXhigh?: boolean;
  /** Whether tool call IDs are required in tool results for this model. */
  requiresToolCallId?: boolean;
  /** Whether OpenAI-style service tiers (priority/flex) apply to this model. */
  supportsServiceTier?: boolean;
  /** Approximate characters per token for estimation fallback. */
  charsPerToken?: number;
}

Add capabilities?: ModelCapabilities to Model<TApi>.

2. Centralize capability declarations in models.ts

Add a CAPABILITY_PATCHES table that declares capabilities for known models by ID pattern. Applied once at module load. supportsXhigh() reads model.capabilities?.supportsXhigh — zero model-ID string comparisons at call sites.

3. Fix service-tier.ts

Read model.capabilities?.supportsServiceTier instead of bare.startsWith("gpt-5.4"). Any future model that sets this capability gets service tier support for free.

4. Fix google-shared.ts:requiresToolCallId()

Read model.capabilities?.requiresToolCallId instead of checking for claude- and gpt-oss- prefixes. Models that cross-deploy (Claude on Google, etc.) declare this in their model definitions.


What this PR does NOT include

  • Token counter tokenizer (gpt-4o tiktoken as universal fallback is pragmatic; separate concern)
  • onboarding-service.ts endpoint hardcoding (separate concern, different surface area)
  • native-search.ts Anthropic coupling (separate concern, needs design discussion)
  • model-router.ts tier/cost table (GSD extension, separate PR)

Impact

  • No behavior change for existing users — capability patches preserve current behavior exactly
  • New providers and models can declare capabilities without touching function logic
  • Lays groundwork for all future capability checks to use the same pattern

Note: AI-assisted contribution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    High PriorityrefactorCode restructuring without behavior changetech-debtTechnical debt reduction

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions