Skip to content

feat(provider): add Cloudflare Workers AI integration (#1100)#1178

Open
0xfandom wants to merge 1 commit into
Gitlawb:mainfrom
0xfandom:feat/1100-cloudflare-workers-ai
Open

feat(provider): add Cloudflare Workers AI integration (#1100)#1178
0xfandom wants to merge 1 commit into
Gitlawb:mainfrom
0xfandom:feat/1100-cloudflare-workers-ai

Conversation

@0xfandom
Copy link
Copy Markdown
Contributor

Summary

Adds Cloudflare Workers AI as a first-class OpenAI-compatible provider preset, mirroring the Venice (#1109) / Xiaomi MiMo (#1152) descriptor pattern.

  • New src/integrations/vendors/cloudflare.ts descriptor:
    • classification: 'openai-compatible'
    • Default base URL with literal <ACCOUNT_ID> placeholder — users substitute via /provider baseUrl edit, same shape docs/advanced-setup.md already uses for Azure OpenAI (https://your-resource.openai.azure.com/...)
    • CLOUDFLARE_API_TOKEN env var (OPENAI_API_KEY fallback per the Mistral / MiMo precedent)
    • removeBodyFields: ['store'] — Workers AI rejects unknown OpenAI body fields; mirrors the Mistral / Gemini / Cerebras strip pattern
    • Static catalog with current Workers AI chat models (@cf/meta/llama-3.3-70b-instruct-fp8-fast, @cf/meta/llama-3.1-8b-instruct, @cf/deepseek-ai/deepseek-r1-distill-qwen-32b, @cf/qwen/qwen2.5-coder-32b-instruct)
    • Validation routing on api.cloudflare.com / gateway.ai.cloudflare.com hosts so a user-edited URL maps back to this preset
  • Env mirror plumbing in src/utils/providerProfiles.ts and src/utils/providerProfile.ts so CLOUDFLARE_API_TOKEN flows through the same code paths VENICE_API_KEY / MIMO_API_KEY already use (3 mirror sites + PROFILE_ENV_KEYS / SECRET_ENV_KEYS / ProfileEnv / SecretValueSource)
  • src/utils/providerFlag.ts --provider <name> startup flag now reverse-maps a profile from OPENAI_API_KEY === CLOUDFLARE_API_TOKEN (matches how the other mirrored providers are detected)
  • bun run scripts/generate-integrations-artifacts.ts regenerated integrationArtifacts.generated.ts to include the cloudflare preset / route / vendor entries.
  • Tests in compatibility.test.ts (PRESETS list), provider.test.tsx (buildProfileSaveMessage Cloudflare case), providerProfiles.test.ts (applyProviderProfileToProcessEnv Cloudflare case).
  • Docs row in README providers table + docs/advanced-setup.md section.

Impact

  • user-facing impact: /provider now offers a Cloudflare Workers AI preset. Users with a Cloudflare account get a guided onboarding path instead of having to construct the OpenAI-compatible env vars by hand. Existing users on other providers see no change.
  • developer/maintainer impact: one new vendor descriptor + 3 mirror-site additions in providerProfiles.ts + 1 reverse-map branch in providerFlag.ts. Pattern is byte-for-byte parallel to the recent Venice and Xiaomi MiMo merges.

What this does NOT do (clean follow-ups)

  • Dedicated AI Gateway integration with gateway_id URL templating. Today users can paste a full Gateway URL into OPENAI_BASE_URL and the preset's matchBaseUrlHosts picks gateway.ai.cloudflare.com up — but there's no first-class onboarding for it.
  • Dynamic /models discovery on the Groq feat(groq): dynamic model discovery with mapModel filtering and hybrid catalog #1143 / mapModel pattern. Cloudflare's /v1/models returns the runnable model list; switching the catalog to source: 'hybrid' is a clean drop-in change. Left as a separate PR so this one stays a focused preset add.

Testing

  • bun run build — green
  • focused tests: bun test src/integrations/ src/commands/provider/ src/utils/providerProfiles.test.ts src/utils/providerFlag.test.ts src/utils/providerProfile.test.ts — 303/303 pass
  • bun run smoke — fails on this checkout with Cannot find package '@orama/orama'; reproduces on clean upstream main HEAD 94e8ff3 (pre-existing local env issue, unrelated to this change)

Notes

  • provider/model path tested: existing OpenAI-compatible shim path with removeBodyFields: ['store']. No new transport, no new auth shape.
  • The literal <ACCOUNT_ID> placeholder URL is intentional — Cloudflare's Workers AI endpoint is account-scoped and there's no way to derive the id from credentials alone. The provider description (Replace <ACCOUNT_ID> in the base URL with your Cloudflare account id.) surfaces the substitution requirement in setup.
  • bun run scripts/generate-integrations-artifacts.ts --check succeeds against the regenerated file.

Closes #1100

Adds Cloudflare Workers AI as a first-class OpenAI-compatible provider
preset, modeled on the Venice / Xiaomi MiMo descriptors.

- New `src/integrations/vendors/cloudflare.ts` descriptor:
  - `classification: 'openai-compatible'`
  - Default base URL with literal `<ACCOUNT_ID>` placeholder — users
    substitute via `/provider` baseUrl edit, same shape as the Azure
    OpenAI example already in `docs/advanced-setup.md`
  - `CLOUDFLARE_API_TOKEN` env, with `OPENAI_API_KEY` as fallback
  - `removeBodyFields: ['store']` since Workers AI rejects unknown
    OpenAI body fields (mirrors Mistral / Gemini / Cerebras strip)
  - Static catalog with current Workers AI chat models
    (`@cf/meta/llama-3.3-70b-instruct-fp8-fast`,
    `@cf/meta/llama-3.1-8b-instruct`,
    `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b`,
    `@cf/qwen/qwen2.5-coder-32b-instruct`)
  - Validation routing on `api.cloudflare.com` /
    `gateway.ai.cloudflare.com` hosts so an env-pasted URL maps back
    to the preset
- Env mirror sites in `src/utils/providerProfiles.ts`: mirror api key
  into `CLOUDFLARE_API_TOKEN` when baseUrl contains a Cloudflare host
  (3 sites: same-env check, openAIProfileEnv build, applyEnv).
- `CLOUDFLARE_API_TOKEN` added to `PROFILE_ENV_KEYS` / `SECRET_ENV_KEYS` /
  `ProfileEnv` / `SecretValueSource` in `src/utils/providerProfile.ts`
  so the profile-clean and secret-redact paths know about it.
- `src/utils/providerFlag.ts` `--provider <name>` startup flag now
  detects a Cloudflare profile from `OPENAI_API_KEY ===
  CLOUDFLARE_API_TOKEN` (mirrors how the other host-key mirrors are
  reverse-mapped to their preset id).
- `bun run scripts/generate-integrations-artifacts.ts` regenerated
  `integrationArtifacts.generated.ts` to include the cloudflare preset
  + route + vendor.
- Tests: `compatibility.test.ts` PRESETS list, new
  `buildProfileSaveMessage` Cloudflare case in `provider.test.tsx`,
  new `applyProviderProfileToProcessEnv` Cloudflare case in
  `providerProfiles.test.ts`.
- Docs: README providers table row + `docs/advanced-setup.md` section
  matching the MiMo / Mistral entries.

## Out of scope (clean follow-ups)

- Dedicated AI Gateway integration with `gateway_id` URL templating.
  Today users can still paste a full Gateway URL into `OPENAI_BASE_URL`
  and the preset's `matchBaseUrlHosts` picks `gateway.ai.cloudflare.com`
  up.
- Dynamic `/models` discovery on the Groq Gitlawb#1143 / `mapModel` pattern —
  Cloudflare's `/v1/models` returns the runnable model list and the
  hybrid catalog path drops in cleanly. Left as a separate PR so this
  one stays a focused preset add.

Closes Gitlawb#1100
Copy link
Copy Markdown
Collaborator

@jatmn jatmn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [P2] Narrow Cloudflare gateway URL matching to Workers AI routes
    src/integrations/vendors/cloudflare.ts:52
    Adding gateway.ai.cloudflare.com to matchBaseUrlHosts makes every Cloudflare AI Gateway URL resolve to the new cloudflare route, because resolveRouteIdFromBaseUrl() matches these entries by hostname only. That includes non-Workers-AI Gateway URLs such as https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/openai, which now get Workers-AI-specific runtime metadata and credential precedence (max_tokens, removeBodyFields: ['store'], no Responses format selection, and CLOUDFLARE_API_TOKEN before OPENAI_API_KEY). Please either remove the broad gateway host match or add path-aware matching that only classifies Workers AI/compat Gateway paths as this preset, with coverage that another provider's Cloudflare AI Gateway URL remains custom/OpenAI-compatible.

@Vasanthdev2004
Copy link
Copy Markdown
Collaborator

Blockers

  1. Broad gateway URL matching — Adding gateway.ai.cloudflare.com to matchBaseUrlHosts makes every Cloudflare AI Gateway URL resolve to the Workers AI route, including non-Workers-AI Gateway URLs. This applies Workers-AI-specific runtime metadata and credential precedence to other providers' Gateway URLs.

Non-Blocking

  • Follow-up work identified (AI Gateway integration, dynamic model discovery).

Looks Good

  • Mirrors the Venice / Xiaomi MiMo descriptor pattern
  • Good test coverage (303 tests passing)
  • No new dependencies
  • Clear documentation and setup guidance
  • removeBodyFields: ['store'] for Workers AI compatibility

Verdict: Changes Requested — gateway URL matching needs to be narrowed to Workers AI routes.

@jatmn
Copy link
Copy Markdown
Collaborator

jatmn commented May 17, 2026

please rebase from main, this should fix your smoke issues

@gnanam1990
Copy link
Copy Markdown
Collaborator

Confirmed @jatmn's point against the code: matchBaseUrlHosts includes gateway.ai.cloudflare.com, which is the shared host for all Cloudflare AI Gateway routes (/openai, /anthropic, etc.), so any AI Gateway URL — not just Workers AI — resolves to this preset and inherits CLOUDFLARE_API_TOKEN-before-OPENAI_API_KEY precedence and the store body-field strip. Path-aware matching (only Workers-AI/compat Gateway paths) or dropping the broad gateway host would fix it. The added test covers the save-message label but not this over-match case, so adding coverage where another provider's CF AI Gateway URL stays generic/OpenAI-compatible would lock the fix in. The rest of the integration (descriptor shape, static catalog, profile env wiring) looks consistent with the MiMo/Venice pattern. Thanks — happy to take another pass after that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Cloudflare AI

4 participants