Migrate from @airegistry/vercel-gateway package to runtime model fetching - #8
Merged
Merged
Conversation
Replace the @airegistry/vercel-gateway package with runtime model fetching from Vercel AI Gateway, following the chat-js auto-updating models pattern. Models are now fetched server-side via unstable_cache (1h TTL, tag-based revalidation) and passed through a per-instance Zustand store via React context. Adds a gateway provider abstraction so additional gateways can be plugged in, plus a generated snapshot fallback and a fetch:models script. Removes the published package, changesets, and release workflow. https://claude.ai/code/session_01EorGcQU6Zoe1UQkx5mnqkh
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedToo many files! This PR contains 209 files, which is 59 over the limit of 150. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (17)
📒 Files selected for processing (209)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
7 issues found across 208 files
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="apps/website/app/(models)/models/models-store-context.tsx">
<violation number="1" location="apps/website/app/(models)/models/models-store-context.tsx:238">
P2: `temperatureControl` filters are ignored in `computeResults`, so toggling that filter won’t change the results even though the UI counts it as active. Add the missing filter check (or remove the feature from state if it’s no longer supported).</violation>
</file>
<file name="apps/website/app/(models)/models/models-constants.ts">
<violation number="1" location="apps/website/app/(models)/models/models-constants.ts:20">
P2: Avoid defaulting missing input pricing to 0; it incorrectly forces the min range to 0 when pricing is absent.</violation>
<violation number="2" location="apps/website/app/(models)/models/models-constants.ts:23">
P2: Avoid defaulting missing output pricing to 0; it will skew the output pricing range to 0 when pricing is absent.</violation>
</file>
<file name="apps/website/lib/ai/to-model-data.ts">
<violation number="1" location="apps/website/lib/ai/to-model-data.ts:13">
P2: Embedding models will be marked as not accepting text input because input.text only checks for "language". If UI or filtering logic relies on input.text, embedding models could be incorrectly excluded. Consider treating embedding models as text input.</violation>
</file>
<file name="apps/website/app/(models)/models/wide-model-details.tsx">
<violation number="1" location="apps/website/app/(models)/models/wide-model-details.tsx:94">
P2: Don’t default missing input pricing to $0; show "--" (or another explicit unknown marker) when pricing.input is absent to avoid misrepresenting the model as free.</violation>
<violation number="2" location="apps/website/app/(models)/models/wide-model-details.tsx:102">
P2: Don’t default missing output pricing to $0; display an explicit unknown value when pricing.output is absent to avoid misleading pricing data.</violation>
</file>
<file name="apps/website/app/(models)/models/gateway-model-card.tsx">
<violation number="1" location="apps/website/app/(models)/models/gateway-model-card.tsx:200">
P2: Don’t coerce missing input pricing to "0". It causes unknown prices to be displayed as free instead of the "--" placeholder returned for undefined values.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| } | ||
| if (f.features.reasoning && !m.reasoning) return false; | ||
| if (f.features.toolCall && !m.toolCall) return false; | ||
| return true; |
There was a problem hiding this comment.
P2: temperatureControl filters are ignored in computeResults, so toggling that filter won’t change the results even though the UI counts it as active. Add the missing filter check (or remove the feature from state if it’s no longer supported).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/models-store-context.tsx, line 238:
<comment>`temperatureControl` filters are ignored in `computeResults`, so toggling that filter won’t change the results even though the UI counts it as active. Add the missing filter check (or remove the feature from state if it’s no longer supported).</comment>
<file context>
@@ -173,363 +157,371 @@ const computeActiveFiltersCount = (f: FilterState): number => {
+ }
+ if (f.features.reasoning && !m.reasoning) return false;
+ if (f.features.toolCall && !m.toolCall) return false;
+ return true;
});
</file context>
| .map((m) => Number.parseFloat(m.pricing.input ?? "0") * 1_000_000) | ||
| .filter((n) => Number.isFinite(n)); | ||
| const outputPrices = allModels | ||
| .map((m) => Number.parseFloat(m.pricing.output ?? "0") * 1_000_000) |
There was a problem hiding this comment.
P2: Avoid defaulting missing output pricing to 0; it will skew the output pricing range to 0 when pricing is absent.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/models-constants.ts, line 23:
<comment>Avoid defaulting missing output pricing to 0; it will skew the output pricing range to 0 when pricing is absent.</comment>
<file context>
@@ -1,35 +1,47 @@
+ .map((m) => Number.parseFloat(m.pricing.input ?? "0") * 1_000_000)
+ .filter((n) => Number.isFinite(n));
+ const outputPrices = allModels
+ .map((m) => Number.parseFloat(m.pricing.output ?? "0") * 1_000_000)
+ .filter((n) => Number.isFinite(n));
+
</file context>
| .map((m) => m.max_tokens) | ||
| .filter((n): n is number => typeof n === "number" && Number.isFinite(n)); | ||
| const inputPrices = allModels | ||
| .map((m) => Number.parseFloat(m.pricing.input ?? "0") * 1_000_000) |
There was a problem hiding this comment.
P2: Avoid defaulting missing input pricing to 0; it incorrectly forces the min range to 0 when pricing is absent.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/models-constants.ts, line 20:
<comment>Avoid defaulting missing input pricing to 0; it incorrectly forces the min range to 0 when pricing is absent.</comment>
<file context>
@@ -1,35 +1,47 @@
+ .map((m) => m.max_tokens)
+ .filter((n): n is number => typeof n === "number" && Number.isFinite(n));
+ const inputPrices = allModels
+ .map((m) => Number.parseFloat(m.pricing.input ?? "0") * 1_000_000)
+ .filter((n) => Number.isFinite(n));
+ const outputPrices = allModels
</file context>
| toolCall: tags.includes("tool-use"), | ||
| input: { | ||
| image: tags.includes("vision") || model.type === "image", | ||
| text: model.type === "language", |
There was a problem hiding this comment.
P2: Embedding models will be marked as not accepting text input because input.text only checks for "language". If UI or filtering logic relies on input.text, embedding models could be incorrectly excluded. Consider treating embedding models as text input.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/lib/ai/to-model-data.ts, line 13:
<comment>Embedding models will be marked as not accepting text input because input.text only checks for "language". If UI or filtering logic relies on input.text, embedding models could be incorrectly excluded. Consider treating embedding models as text input.</comment>
<file context>
@@ -0,0 +1,25 @@
+ toolCall: tags.includes("tool-use"),
+ input: {
+ image: tags.includes("vision") || model.type === "image",
+ text: model.type === "language",
+ pdf: tags.includes("file-input"),
+ audio: false,
</file context>
Comment on lines
+102
to
103
| ? `$${(Number.parseFloat(model.pricing.output ?? "0") * 1_000_000).toFixed(2)}/M tokens` | ||
| : "--" |
There was a problem hiding this comment.
P2: Don’t default missing output pricing to $0; display an explicit unknown value when pricing.output is absent to avoid misleading pricing data.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/wide-model-details.tsx, line 102:
<comment>Don’t default missing output pricing to $0; display an explicit unknown value when pricing.output is absent to avoid misleading pricing data.</comment>
<file context>
@@ -100,15 +91,15 @@ export function WideModelDetails({
value={
model
- ? `$${(Number.parseFloat(model.pricing.output) * 1_000_000).toFixed(2)}/M tokens`
+ ? `$${(Number.parseFloat(model.pricing.output ?? "0") * 1_000_000).toFixed(2)}/M tokens`
: "--"
}
</file context>
Suggested change
| ? `$${(Number.parseFloat(model.pricing.output ?? "0") * 1_000_000).toFixed(2)}/M tokens` | |
| : "--" | |
| ? model.pricing.output | |
| ? `$${(Number.parseFloat(model.pricing.output) * 1_000_000).toFixed(2)}/M tokens` | |
| : "--" | |
| : "--" |
Comment on lines
+94
to
95
| ? `$${(Number.parseFloat(model.pricing.input ?? "0") * 1_000_000).toFixed(2)}/M tokens` | ||
| : "--" |
There was a problem hiding this comment.
P2: Don’t default missing input pricing to $0; show "--" (or another explicit unknown marker) when pricing.input is absent to avoid misrepresenting the model as free.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/wide-model-details.tsx, line 94:
<comment>Don’t default missing input pricing to $0; show "--" (or another explicit unknown marker) when pricing.input is absent to avoid misrepresenting the model as free.</comment>
<file context>
@@ -100,15 +91,15 @@ export function WideModelDetails({
value={
model
- ? `$${(Number.parseFloat(model.pricing.input) * 1_000_000).toFixed(2)}/M tokens`
+ ? `$${(Number.parseFloat(model.pricing.input ?? "0") * 1_000_000).toFixed(2)}/M tokens`
: "--"
}
</file context>
Suggested change
| ? `$${(Number.parseFloat(model.pricing.input ?? "0") * 1_000_000).toFixed(2)}/M tokens` | |
| : "--" | |
| ? model.pricing.input | |
| ? `$${(Number.parseFloat(model.pricing.input) * 1_000_000).toFixed(2)}/M tokens` | |
| : "--" | |
| : "--" |
| Input{" "} | ||
| <span className="font-medium text-foreground"> | ||
| {formatUsdPerMTokens(model.pricing.input)} | ||
| {formatUsdPerMTokens(model.pricing.input ?? "0")} |
There was a problem hiding this comment.
P2: Don’t coerce missing input pricing to "0". It causes unknown prices to be displayed as free instead of the "--" placeholder returned for undefined values.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/website/app/(models)/models/gateway-model-card.tsx, line 200:
<comment>Don’t coerce missing input pricing to "0". It causes unknown prices to be displayed as free instead of the "--" placeholder returned for undefined values.</comment>
<file context>
@@ -205,33 +197,22 @@ function ModelMetaRow({ model }: { model: ModelDefinition }) {
Input{" "}
<span className="font-medium text-foreground">
- {formatUsdPerMTokens(model.pricing.input)}
+ {formatUsdPerMTokens(model.pricing.input ?? "0")}
</span>
</span>
</file context>
Suggested change
| {formatUsdPerMTokens(model.pricing.input ?? "0")} | |
| {formatUsdPerMTokens(model.pricing.input)} |
Match the chat-js pattern: render provider icons via a next/image pointing at https://models.dev/logos/{owned_by}.svg instead of the @lobehub/icons component library and the local /public/providers SVGs. This drops the hard-coded provider switch so icons automatically appear for any new gateway model. @lobehub/icons is removed from dependencies and models.dev is whitelisted in next.config image remotePatterns. https://claude.ai/code/session_01EorGcQU6Zoe1UQkx5mnqkh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR removes the
@airegistry/vercel-gatewaypackage dependency and replaces it with runtime model fetching from the Vercel AI Gateway API. Instead of maintaining a static, pre-generated list of models, the application now dynamically fetches and caches model data at runtime.Key Changes
Removed static model registry: Deleted the entire
packages/vercel-gatewaypackage including:models.generated.ts,model-extra.generated.ts,model-features.generated.ts)Added runtime model fetching: Created new gateway abstraction layer:
lib/ai/gateways/gateway-provider.ts- Interface for gateway implementationslib/ai/gateways/vercel-gateway.ts- Vercel AI Gateway implementation with HTTP fetchinglib/ai/gateways/fallback-models.ts- Fallback models for offline/error scenarioslib/ai/models.ts- Cached model fetching with Next.jsunstable_cacheUpdated model data handling:
lib/ai/ai-gateway-models-schemas.tswith Zod schemas for API response validationlib/ai/model-data.tstype definition for internal model representationlib/ai/to-model-data.tsfor converting API responses to internal formatlib/ai/active-gateway.tsfor gateway selection logicUpdated model constants: Modified
models-constants.tsto compute range limits dynamically from fetched models instead of using pre-generated constantsUpdated all model references: Changed imports throughout the app from
@airegistry/vercel-gatewayto use the new local gateway implementationsAdded model sync script: Created
scripts/fetch-models.tsfor manual model data fetching and cachingRemoved build infrastructure: Deleted changesets configuration, release workflows, and model sync scripts that are no longer needed
Implementation Details
unstable_cachefor performancehttps://claude.ai/code/session_01EorGcQU6Zoe1UQkx5mnqkh
Summary by cubic
Migrates the website from the static
@airegistry/vercel-gatewaypackage to runtime model fetching from the Vercel AI Gateway with caching and a fallback snapshot, and switches provider logos to a CDN so new providers appear automatically.New Features
lib/ai/models.generated.tswithpnpm --filter website fetch:modelsto pre-generate it.fetchModels(),getModelById(),getProviders(), plus a per-requestModelsProviderstore; filter ranges computed from live data.ModelDatatype.models.devCDN vianext/image, removing the hard-coded icon switch and local SVGs.Migration
@airegistry/vercel-gatewayimports withfetchModels()/getModelById()and wrap pages withModelsProvider.AI_GATEWAY_API_KEY(orVERCEL_OIDC_TOKEN); optionally setAI_GATEWAY=vercel. Usepnpm --filter website fetch:modelsto generate a build-time fallback when offline.models.dev; remove local/public/providers/*.svgand@lobehub/icons, and allowmodels.devinnext.config.tsimages.remotePatterns.Written for commit 49f127b. Summary will update on new commits.