feat: add OGX gateway deployment manifests for Claude Code#177
feat: add OGX gateway deployment manifests for Claude Code#177hmoghani wants to merge 6 commits into
Conversation
Add parameterized deployment manifests for OGX as a standalone API gateway between Claude Code and vLLM. OGX provides the Anthropic Messages API (/v1/messages) passthrough to vLLM's OpenAI-compatible backend. Includes ConfigMap, Deployment, Service, NetworkPolicy, and README with configuration reference and known issues.
|
Warning Review limit reached
More reviews will be available in 33 minutes and 46 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more credits in the billing tab to continue. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR adds a complete OGX (API gateway) deployment package for Claude Code integration with vLLM on Kubernetes. It includes documentation, configuration, and all required manifests to deploy a single-replica OGX service that translates Anthropic Messages API requests to OpenAI-compatible vLLM calls. ChangesOGX Deployment Configuration
🎯 2 (Simple) | ⏱️ ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@agents/claude-code/ogx/ogx-network-policy.yaml`:
- Around line 26-28: The NetworkPolicy currently uses namespaceSelector to allow
all pods in <NAMESPACE> to access OGX on port 8321; change the ingress rule to
restrict traffic to only the Claude Code and router pods by replacing the broad
namespaceSelector with a podSelector that matches the specific labels used by
those deployments (e.g., the Claude Code pod label and the router pod label) or
by using both namespaceSelector + podSelector (namespace: <NAMESPACE> and
podSelector: matchLabels: { <claude-label-key>: <claude-label-value> } and
another peer for the router); ensure the ingress rule still targets port 8321
for the OGX service and remove the open namespace-wide matchLabels entry under
namespaceSelector.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: 29b76491-e767-4b0f-9fbf-d70fa55353e8
📒 Files selected for processing (5)
agents/claude-code/ogx/README.mdagents/claude-code/ogx/ogx-configmap.yamlagents/claude-code/ogx/ogx-deployment.yamlagents/claude-code/ogx/ogx-network-policy.yamlagents/claude-code/ogx/ogx-service.yaml
OGX uses native /v1/messages passthrough when vLLM supports it, not Anthropic-to-OpenAI translation. Updated configmap, deployment, and README descriptions to reflect this.
|
@hmoghani LGTM. I added one more comment. |
aakankshaduggal
left a comment
There was a problem hiding this comment.
Looks good overall — well-documented, properly parameterized, and the OGX_PORT service naming collision catch is a nice find.
A few items:
-
Missing
seccompProfilein the deployment security context — restricted SCC requiresseccompProfile: RuntimeDefaultalongsiderunAsNonRootanddrop ALL. Should add:seccompProfile: type: RuntimeDefault
-
api_token: fakein the configmap — works because vLLM doesn't validate tokens by default, but someone copying this for a setup with--api-keyenabled would get silently broken auth. A one-line comment like# vLLM default: no auth. Replace with a real token or Secret reference if vLLM is configured with --api-keywould help. -
Minor (non-blocking): no pod-level
securityContext— only the container-level context is set. For full restricted SCC compliance you'd typically also setrunAsNonRootandseccompProfileatspec.template.spec.securityContext. Not blocking for a starter kit.
Reviewed by @aakankshaduggal with Claude Code
- Add seccompProfile: RuntimeDefault at both pod and container level - Add pod-level securityContext (runAsNonRoot, seccompProfile) - Add comment on api_token explaining when to replace it
I updated the manifests. Thanks for the review. |
Summary
Add parameterized OGX deployment manifests for the Claude Code + vLLM setup. The existing deployment README already documents the OGX+vLLM path (env vars, model aliasing, vLLM flags). This PR adds the missing OGX infrastructure manifests that the README references.
What's Included
OGX Manifests (
agents/claude-code/ogx/)ogx-configmap.yaml— OGX runtime configuration enabling the Anthropic Messages API (/v1/messages) passthrough viainline::builtinproviderogx-deployment.yaml— Standalone OGX Deployment (no LlamaStack operator required), with security hardeningogx-service.yaml— ClusterIP Service (namedogx-serviceto avoid Kubernetes env var collision)ogx-network-policy.yaml— NetworkPolicy allowing ingress on port 8321README.md— Configuration reference, quick start, verification steps, and known issuesAll manifests use parameterized placeholders (
<VLLM_SERVICE>,<NAMESPACE>,<VLLM_PORT>,<MODEL_ID>).Key Findings
messagesAPI andinline::builtinprovider in its config to enable/v1/messages— the default vLLM-only config does not expose this endpointogxdue to a Kubernetes env var collision (OGX_PORT)Test plan
/v1/health,/v1/models,/v1/messages(non-streaming + streaming)