feat: add generic provider discovery APIs#283
Open
sozercan wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes provider discovery dynamic by moving provider catalog metadata (display name, description, default namespace, capabilities, health probe hints) out of InferenceProviderConfig.spec and into annotations, exposing it through new first-class /api/providers[/:id] endpoints, and migrating both the backend health/install logic and frontend Settings/Deployment UIs to consume it. It refs #276.
Changes:
- New backend
lib/providers.tsparses provider annotations; new/api/providersroutes plus annotation-driven health probing inkubernetes.ts; deployments now validate engine/mode/source/router against provider capabilities. - Each provider (
kaito,dynamo,kuberay,llmd) emitsairunway.ai/{display-name,description,default-namespace,documentation-url,capabilities,health,installation}annotations and removesSpec.Capabilities; CRD schema and samples dropspec.capabilities. - Frontend drops hardcoded
RuntimeId/RUNTIME_INFO/RUNTIME_ENGINESand now derives runtime metadata, default namespace, and engine compatibility fromRuntimeStatus.capabilities/defaultNamespace/description;useProviders/useProviderDetailsswitch to theprovidersApi.
Show a summary per file
| File | Description |
|---|---|
| shared/types/settings.ts | Adds ProviderCapabilities/ProviderDeploymentDefaults/ProviderHealthConfig and extends ProviderInfo/RuntimeStatus/HelmChart. |
| backend/src/lib/providers.ts | New annotation parser producing ProviderInfo/ProviderDetails. |
| backend/src/routes/providers.ts, routes/index.ts, hono-app.ts | New /api/providers[/:id] endpoints registered as public routes. |
| backend/src/routes/settings.ts, settings.test.ts | Settings response now embeds providers and exposes duplicate /providers[/:id] handlers. |
| backend/src/routes/installation.ts, installation.test.ts, provider-installation-flow.test.ts | Installation logic uses shared provider extractor and annotation-driven health checks instead of provider-specific cases. |
| backend/src/routes/deployments.ts, deployments.test.ts | Adds validateProviderCapabilities for engine/mode/source/router-mode checks. |
| backend/src/services/kubernetes.ts, kubernetes-runtime-status.test.ts | New listInferenceProviderConfigs, annotation-driven checkProviderInstallationStatus, and helpers for CRD/operator-pod probes. |
| backend/src/test/fixtures.ts | Mock InferenceProviderConfig now carries display/capabilities/health annotations. |
| controller/api/v1alpha1/inferenceproviderconfig_types.go, zz_generated.deepcopy.go | Removes Spec.Capabilities, adds new annotation constants and HelmChart fields. |
| controller/internal/controller/modeldeployment_controller.go | Provider capability lookup switched to annotation parsing; selection algorithm gains context-aware variant. |
| controller/internal/gateway/provider_capabilities.go(+test) | Gateway capability resolver reads airunway.ai/capabilities annotation. |
| controller/config/crd/bases/...yaml, deploy/controller.yaml, controller/config/samples/...yaml | CRD schema drops spec.capabilities; samples move metadata into annotations. |
| providers/{kaito,dynamo,kuberay,llmd}/config.go(+tests) | Provider registration writes display/health/capabilities annotations and drops spec capabilities. |
| frontend/src/lib/api.ts, hooks/useSettings.ts, test/mocks/handlers.ts | Frontend switches to providersApi against /api/providers. |
| frontend/src/pages/SettingsPage.tsx | Drops hardcoded RuntimeId map, uses dynamic provider list. |
| frontend/src/components/deployments/DeploymentForm.tsx(+test) | Removes RUNTIME_INFO/RUNTIME_ENGINES; runtime metadata comes from RuntimeStatus.capabilities. |
| docs/{crd-reference,gateway,development}.md, agents.md | Documentation updated to reflect annotation-based capabilities and removed spec.capabilities. |
| package.json, bun.lock, .eslintrc.cjs | Adds ESLint config and parser/plugin devDependencies for react-hooks/react-refresh rules. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Files not reviewed (1)
- controller/api/v1alpha1/zz_generated.deepcopy.go: Language not supported
- Files reviewed: 42/44 changed files
- Comments generated: 4
Comment on lines
+29
to
+47
| .get('/providers', async (c) => { | ||
| logger.debug('Fetching provider list'); | ||
| const providerConfigs = await kubernetesService.listInferenceProviderConfigs(); | ||
|
|
||
| return c.json({ | ||
| providers: providerConfigs.map(extractProviderInfo), | ||
| }); | ||
| }) | ||
| .get('/providers/:id', async (c) => { | ||
| const id = c.req.param('id'); | ||
| logger.debug({ id }, 'Fetching provider details'); | ||
|
|
||
| const config = await kubernetesService.getInferenceProviderConfig(id); | ||
| if (!config) { | ||
| throw new HTTPException(404, { message: `Provider not found: ${id}` }); | ||
| } | ||
|
|
||
| return c.json(extractProviderDetails(config)); | ||
| }) |
Comment on lines
+204
to
210
| // InferenceProviderConfigSpec defines the desired state of InferenceProviderConfig. | ||
| type InferenceProviderConfigSpec struct { | ||
| // capabilities defines what this provider supports | ||
| // +optional | ||
| Capabilities *ProviderCapabilities `json:"capabilities,omitempty"` | ||
|
|
||
| // selectionRules defines rules for auto-selecting this provider | ||
| // Conditions use CEL (Common Expression Language) | ||
| // selectionRules defines rules for auto-selecting this provider. | ||
| // Conditions use CEL (Common Expression Language). | ||
| // +optional | ||
| SelectionRules []SelectionRule `json:"selectionRules,omitempty"` | ||
| } |
Comment on lines
+503
to
+509
| // runSelectionAlgorithm implements the provider selection algorithm. | ||
| func (r *ModelDeploymentReconciler) runSelectionAlgorithm(md *airunwayv1alpha1.ModelDeployment, providers []airunwayv1alpha1.InferenceProviderConfig) (string, string, error) { | ||
| return r.runSelectionAlgorithmWithContext(context.Background(), md, providers) | ||
| } | ||
|
|
||
| func (r *ModelDeploymentReconciler) runSelectionAlgorithmWithContext(ctx context.Context, md *airunwayv1alpha1.ModelDeployment, providers []airunwayv1alpha1.InferenceProviderConfig) (string, string, error) { | ||
| logger := log.FromContext(ctx) |
Comment on lines
+314
to
+336
| async function validateProviderCapabilities(config: DeploymentConfig): Promise<void> { | ||
| if (!config.provider) { | ||
| return; | ||
| } | ||
|
|
||
| const providerConfig = await kubernetesService.getInferenceProviderConfig(config.provider); | ||
| if (!providerConfig) { | ||
| throw new HTTPException(404, { message: `Provider not found: ${config.provider}` }); | ||
| } | ||
|
|
||
| const provider = extractProviderDetails(providerConfig); | ||
| const capabilities = provider.capabilities ?? { | ||
| engines: [], | ||
| modes: [], | ||
| modelSources: [], | ||
| routerModes: [], | ||
| }; | ||
|
|
||
| validateSupportedCapability(config.provider, 'engine', config.engine, capabilities.engines); | ||
| validateSupportedCapability(config.provider, 'mode', config.mode, capabilities.modes); | ||
| validateSupportedCapability(config.provider, 'model source', config.modelSource, capabilities.modelSources); | ||
| validateSupportedCapability(config.provider, 'router mode', config.routerMode, capabilities.routerModes); | ||
| } |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 971a1f3b74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| const provider = extractProviderDetails(config); | ||
| const charts = normalizeInstallCharts(providerId, provider.helmCharts); | ||
| const charts = provider.helmCharts; |
There was a problem hiding this comment.
When installing kaito or dynamo, this now passes the raw annotation chart metadata through, but the real provider configs in providers/kaito/config.go and providers/dynamo/config.go do not set preInstallMissingCrds or skipCrds (I checked with rg "PreInstallMissing|SkipCRDs" providers). The removed normalizeInstallCharts used to add those flags for kaito/workspace and dynamo-platform; without them one-click install runs Helm directly instead of applying missing chart CRDs first, so these built-in installs can fail on clusters where their CRDs are absent or conflict with Helm CRD handling.
Useful? React with 👍 / 👎.
sozercan
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
InferenceProviderConfigannotations/providersAPIairunway.ai/capabilitiesmetadata location for custom providersRefs #276
Scope note
This PR makes provider registration/discovery generic and exposes provider metadata/capabilities through the API/UI. Advanced deployment UX is still partially provider-id based today, so custom providers can be discovered and surfaced generically, but Dynamo/KAITO-specific UI behavior has not been fully generalized in this PR.
Follow-up work
selectedRuntime === 'dynamo'andselectedRuntime === 'kaito'into capability-based helpers, for exampleproviderSupports('disaggregatedServing'),providerSupports('routerModes'),providerSupports('storageVolumes'), andproviderSupports('aiConfigurator').selectedProvider.capabilities.featuresinstead of runtime/provider ids, especially infrontend/src/components/deployments/DeploymentForm.tsx.frontend/src/hooks/useMetrics.ts,backend/src/services/metrics.ts, andshared/types/deployment.tsas needed.Tests
bun run --filter '@airunway/frontend' test src/hooks/useSettings.test.tsx