-
-
Notifications
You must be signed in to change notification settings - Fork 376
Description
Summary
The codebase has a systemic pattern of checking capability support by parsing model ID strings (model.id.includes(\"gpt-5.4\"), model.id.startsWith(\"claude-\")) or hardcoding provider names (model.provider === \"xai\") at call sites. This violates the provider-agnostic principle in VISION.md — any new model or provider that doesn't match a hardcoded pattern breaks silently.
The correct fix: declare capabilities in Model<TApi> metadata so capability checks read from the model object, not from ID strings.
Findings (full audit)
HIGH — breaks provider-agnostic principle
| File | Issue |
|---|---|
packages/pi-ai/src/models.ts:74-84 |
supportsXhigh() hardcodes gpt-5.2/5.3/5.4 and opus-4-6/4.6 ID patterns |
src/cli.ts:389-393 |
Default model fallback hardcodes OpenAI-first then Anthropic priority order |
src/web/onboarding-service.ts:336-379 |
Provider API endpoints and version headers hardcoded in core onboarding service |
src/resources/extensions/gsd/service-tier.ts:34-38 |
supportsServiceTier() gates on gpt-5.4 prefix — no extension point for other providers |
MEDIUM — cross-provider leakage in provider files
| File | Issue |
|---|---|
packages/pi-ai/src/providers/google-shared.ts:70 |
requiresToolCallId() checks for claude- and gpt-oss- prefixes inside a Google provider file |
packages/pi-ai/src/providers/google-gemini-cli.ts:207,919 |
Google provider explicitly detects Claude models by name |
packages/pi-ai/src/providers/amazon-bedrock.ts:229,687 |
Bedrock provider hardcodes anthropic.claude / anthropic/claude pattern checks |
packages/pi-ai/src/providers/openai-completions.ts:393 |
OpenAI handler special-cases Anthropic models routed via OpenRouter |
src/resources/extensions/gsd/token-counter.ts |
Hardcoded chars-per-token ratios per provider; falls back to gpt-4o tiktoken as universal tokenizer |
src/resources/extensions/gsd/model-router.ts |
Static model-to-tier and model-to-cost tables — adding new models requires code changes |
src/resources/extensions/gsd/prompt-cache-optimizer.ts:191-198 |
Cache efficiency multipliers hardcoded per provider |
src/resources/extensions/search-the-web/native-search.ts |
Deeply coupled to Anthropic's web_search_20250305 tool |
LOW / Cosmetic
| File | Issue |
|---|---|
src/resources/extensions/gsd/export-html.ts |
Shortens claude- prefixes in display but not other providers |
src/resources/extensions/gsd/preferences-models.ts |
Default model profiles are all Claude (overridable, but still) |
Root cause
The codebase conflates model name patterns with provider capabilities. The Model<TApi> type already has compat fields for OpenAI-compatible APIs — the pattern exists. It needs to be extended with a general capabilities field so any model or provider can declare what it supports without requiring changes to call-site logic.
Proposed fix
1. Add ModelCapabilities to types.ts
export interface ModelCapabilities {
/** Whether the model supports xhigh thinking level. */
supportsXhigh?: boolean;
/** Whether tool call IDs are required in tool results for this model. */
requiresToolCallId?: boolean;
/** Whether OpenAI-style service tiers (priority/flex) apply to this model. */
supportsServiceTier?: boolean;
/** Approximate characters per token for estimation fallback. */
charsPerToken?: number;
}Add capabilities?: ModelCapabilities to Model<TApi>.
2. Centralize capability declarations in models.ts
Add a CAPABILITY_PATCHES table that declares capabilities for known models by ID pattern. Applied once at module load. supportsXhigh() reads model.capabilities?.supportsXhigh — zero model-ID string comparisons at call sites.
3. Fix service-tier.ts
Read model.capabilities?.supportsServiceTier instead of bare.startsWith("gpt-5.4"). Any future model that sets this capability gets service tier support for free.
4. Fix google-shared.ts:requiresToolCallId()
Read model.capabilities?.requiresToolCallId instead of checking for claude- and gpt-oss- prefixes. Models that cross-deploy (Claude on Google, etc.) declare this in their model definitions.
What this PR does NOT include
- Token counter tokenizer (gpt-4o tiktoken as universal fallback is pragmatic; separate concern)
onboarding-service.tsendpoint hardcoding (separate concern, different surface area)native-search.tsAnthropic coupling (separate concern, needs design discussion)model-router.tstier/cost table (GSD extension, separate PR)
Impact
- No behavior change for existing users — capability patches preserve current behavior exactly
- New providers and models can declare capabilities without touching function logic
- Lays groundwork for all future capability checks to use the same pattern
Note: AI-assisted contribution.