Skip to content

models: expand native capability readers with conservative fallback #5575

Description

@RaresKeY

Prerequisites

  • I searched open issues and this has not already been proposed.
  • I searched discussions and this is not already being debated there.
  • This is a concrete, actionable proposal — not a vague feature request.

Area

API / Backend

Problem or Motivation

Issue #4145 and PR #2739 intentionally bootstrap the canonical model capability schema with a small tested provider set. They do not own broad provider coverage. Odysseus already supports more hosted providers and local serving engines, but their model-list payloads differ per provider and sometimes per model.

Treating common field names such as type, pipeline_tag, capabilities, supported_parameters, or max_model_len as a universal contract would turn compatible-looking payloads into incorrect capability claims. Conversely, discarding an unknown provider entirely prevents basic model inventory from working while Odysseus catches up with a new API revision.

The next tracker slice should expand provider-native readers while defining a small, explicit fallback that preserves identity but cannot enable capability-gated behavior.

Proposed Solution

Add a focused follow-up to #2739 that:

  • emits one lean canonical model record with schema version, provider/model identity, family/task, modalities, features, limits, controls, and compact evidence;
  • recognizes discriminating native catalog shapes and maps only provider-owned fields for Anthropic, Mistral, GitHub Copilot, ChatGPT Subscription, SGLang, Hugging Face Hub, Cohere, and other tested shapes;
  • keeps provider identity separate from per-model capability, including when a provider serves models with different capabilities;
  • preserves an explicitly configured but previously unknown provider ID;
  • marks generic data[], models[], and bare-list envelopes as explicit fallback inventory and recovers only id/name/model/key/slug;
  • disables all generic promotion from task, modality, capability, parameter, and limit-looking fields;
  • keeps provider request/response adapters and model-specific runtime quirks out of the catalog detector;
  • emits a bounded normalization summary through the existing LOG_LEVEL=DEBUG toggle without logging model IDs, raw payloads, URLs, or credentials;
  • adds positive native-shape fixtures and neighboring negative/fallback regressions.

Parent tracker

Part of #2737.

Depends on

Implementation PR

#5576 is the current stacked draft implementation. It targets dev and depends on #2739; after #2739 lands, the branch will be rebased so the PR comparison contains only the follow-up.

Scope

In scope:

  • tested provider-native catalog readers;
  • provider identity distinct from per-model capability;
  • a lean, versioned canonical model record;
  • explicit identity-only fallback inventory;
  • conservative provider detection;
  • opt-in normalization diagnostics through existing logging configuration;
  • positive native-shape and negative fallback regression tests.

Out of scope:

  • model-name, prefix, regex, or default-port inference;
  • generic promotion from capability-looking JSON fields;
  • provider request/response adapters;
  • model-specific runtime quirks;
  • registry/admin override layering;
  • durable probe storage;
  • /api/models capability exposure;
  • model-picker UI or routing changes.

Acceptance Criteria

  • Unknown or malformed payloads fail soft.
  • Default local ports never identify a provider.
  • A generic record with capability-looking fields remains canonical family: unknown with no features, limits, or controls.
  • Native fields are interpreted only after a discriminating provider shape or explicit provider identity selects that reader.
  • Serialized canonical output has one stable versioned shape and no duplicate capability/evidence trees.
  • Debug diagnostics are opt-in through existing logging configuration and exclude payload identity and raw values.

Alternatives Considered

  • Extend models: bootstrap capability schema and shape readers #4145/feat(models): define capability schema and readers #2739 to cover every provider. That would erase the bootstrap boundary in the current PR and make its review surface much larger.
  • Infer capabilities from model names, regexes, or common JSON field names. These are inconsistent across providers and local serving engines.
  • Keep a large request/response path registry beside catalog detection. Runtime adapters already own those paths; duplicating them creates two sources of truth without a consumer.
  • Drop unknown providers entirely. Identity-only inventory is useful and safe when it is explicitly marked as fallback.

Prior Art / Related Issues

Are you willing to implement this?

Yes — draft PR #5576 is open as a stacked follow-up to #2739.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestproviderModel/inference provider integrationsready for reviewDescription complete — ready for maintainer review

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions