Status: experimental. Read "What was tested" before relying on any of this.
LiteLLM is a separate, optional Python service that exposes many model providers behind one OpenAI-style endpoint. Phantom does not depend on it and does not ship it. This page shows how to run LiteLLM so that it never holds a real provider key: LiteLLM sends a phm_ placeholder, and Phantom's local proxy injects the real key.
Phantom answers "who holds the real key". LiteLLM answers "which model and which budget". They are complementary. Without Phantom, LiteLLM reads real keys from its config, environment, or database, and that is exactly what a compromised LiteLLM process steals.
- Minimum: LiteLLM 1.83.0 or newer (see "Pin and verify LiteLLM first").
- Config:
examples/litellm/config.yaml. It holds no key:api_base,api_keyandmaster_keyare allos.environ/...references, andallow_client_side_credentialsisfalse. A test pins this and checks thatphantom doctorrecognizes the file as routed through Phantom (shipped_example_config_is_detected_safe_and_value_freeincrates/phantom-cli/src/commands/doctor_litellm.rs). - Command, after
phantom add OPENAI_API_KEY:
LITELLM_MASTER_KEY=sk-<your own random value> \
phantom exec -- litellm --config examples/litellm/config.yaml --host 127.0.0.1 --port 4000- Why
phantom execworks here: LiteLLM uses Phantom's built-inopenairoute through the exportedOPENAI_BASE_URL. It does not need a custom route, whichphantom execrefuses (see "LiteLLM as an upstream" below). Putting LiteLLM upstream of Phantom is the other direction and needsphantom start. - Not tested end to end: this exact command with a real LiteLLM and a real provider. What was tested is listed under "What was tested".
- PyPI releases
litellm==1.82.7and1.82.8were malicious. LiteLLM states they were live on 2026-03-24 from 10:39 UTC for about 40 minutes and have been removed from PyPI. They harvested environment variables, SSH keys and cloud credentials. Source: https://docs.litellm.ai/blog/security-update-march-2026 (checked 2026-10-18). GitHub advisory GHSA-5mg7-485q-xm76 (critical) covers>=1.82.7, <=1.82.8: https://github.com/advisories/GHSA-5mg7-485q-xm76. - LiteLLM's same post says the official proxy Docker image was not affected, because it pins its dependencies.
- LiteLLM resumed releases at
v1.83.0from a rebuilt pipeline. Use1.83.0or newer, and re-check the advisory list for the version you pick, because the proxy has had serious advisories since (https://github.com/BerriAI/litellm/security-advisories). The example below was run with1.103.5. - Never run an unpinned
pip install litellmon a machine that holds real secrets. - Preferred: the official image, verified with cosign. LiteLLM signs images from
v1.83.0-nightlyonward with one key introduced in commit0112e53:
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:<release-tag>- Alternative: a hash-pinned install (
pip install --require-hashes -r requirements.txtwith every transitive dependency hashed, wheels only) in a throwaway virtualenv, started with a minimal environment.
- Optional service only. Phantom and your project must work without it.
- Loopback only: start it with
--host 127.0.0.1. Do not publish the port. - Keep real provider keys out of LiteLLM entirely. Do not use
.envfiles,os.environ/with a real key, or LiteLLM's secret-manager integrations. - Never use
phantom revealto hand a key to LiteLLM (or to anything else in this setup). It prints the real value, which puts it in LiteLLM's environment, your shell history or the clipboard, and undoes the point of the gateway. LiteLLM needs only thephm_placeholder thatphantom execexports. - Set
allow_client_side_credentials: falseundergeneral_settings. LiteLLM documents this as the workaround for GHSA-3cv6-jpf6-8222, where a caller could redirect the proxy's outbound call and make it send its provider credentials to another host. - No Enterprise features. Do not set
LITELLM_LICENSE. Installing theproxyextra also installs alitellm-enterprisepackage; I did not verify what it enables, and this setup configures no Enterprise feature. - Same-user processes on your machine can still reach Phantom's loopback proxy and see its session token. Phantom's threat model says so, and running LiteLLM does not change it.
- Store the provider key in Phantom once, from a trusted terminal (hidden prompt). Your project's
.envthen holds aphm_placeholder, not the key:
phantom add OPENAI_API_KEY- Under
phantom exec, the child process getsOPENAI_API_KEYas a sessionphm_placeholder andOPENAI_BASE_URLas the local proxy URL (http://127.0.0.1:<port>/openai/_phantom/<session token>/). LiteLLM reads both from the environment. This isexamples/litellm/config.yaml:
model_list:
- model_name: gpt-gateway
litellm_params:
model: openai/gpt-4o-mini
api_base: os.environ/OPENAI_BASE_URL # the Phantom proxy, set by phantom exec
api_key: os.environ/OPENAI_API_KEY # a phm_ placeholder, never a real key
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
allow_client_side_credentials: false- Start LiteLLM under Phantom, bound to loopback, with a master key that only the LiteLLM process and your agents know:
LITELLM_MASTER_KEY=sk-<your own random value> \
phantom exec -- litellm --config examples/litellm/config.yaml --host 127.0.0.1 --port 4000- Point an agent at
http://127.0.0.1:4000with the LiteLLM master key (or a virtual key) as its credential. Never give an agent a provider key or the Phantom proxy token. See "Connect an agent" below.
The agent talks to LiteLLM, and LiteLLM talks to Phantom's proxy with a phm_ placeholder. In every case the agent's credential is a LiteLLM key, never a provider key or the Phantom proxy token, and LiteLLM stays on 127.0.0.1. Without the gateway nothing changes: each agent's direct Phantom setup stays the default.
The agent-side settings below come from LiteLLM's documentation. Phantom has not tested any of these agents against LiteLLM; the Phantom side was tested only with OpenAI-style chat completions (see "What was tested").
- Claude Code: set
ANTHROPIC_BASE_URLto the LiteLLM proxy andANTHROPIC_AUTH_TOKENto a LiteLLM key (https://docs.litellm.ai/docs/tutorials/claude_responses_api). Direct setup: Claude Code. - Codex: a custom provider in
~/.codex/config.tomlwithbase_urlset to the LiteLLM proxy,env_key = "LITELLM_API_KEY"andwire_api = "responses"(https://docs.litellm.ai/docs/tutorials/openai_codex). Put a LiteLLM key inLITELLM_API_KEY. The Responses API path was not tested. Direct setup: Codex. - Gemini CLI: set
GOOGLE_GEMINI_BASE_URLto the LiteLLM proxy andGEMINI_API_KEYto a LiteLLM key (https://docs.litellm.ai/docs/tutorials/litellm_gemini_cli). - Cursor: not supported with a loopback gateway. LiteLLM's documentation says its Cursor integration (an overridden OpenAI base URL) makes Cursor's servers call the proxy, so it must be reachable from the internet, and calls it best effort (https://docs.litellm.ai/docs/tutorials/cursor_integration). Do not expose a keyholding proxy for Cursor. Direct setup: Cursor.
Manual smoke test with a real LiteLLM, run on 2026-10-18 with examples/litellm/verify.py (LITELLM_BIN=<path to litellm> python3 examples/litellm/verify.py):
- LiteLLM 1.103.5 installed from PyPI wheels into a fresh virtualenv, started with a cleared environment and a temporary HOME.
- The real Phantom proxy code, started by a test harness (
crates/phantom-proxy/examples/litellm_loopback_harness.rs) with anopenairoute whose upstream is a plain-HTTP mock on 127.0.0.1. The harness is needed because routes built from.phantom.tomlare alwayshttps://, so a mock upstream cannot be reached through the CLI path. It copies thephantom execproxy settings (session token in the URL path,/v1added for the OpenAI route). - One non-streaming
POST /v1/chat/completionsthrough LiteLLM. The mock recorded the request. Checks that passed: the mock received the injected stand-in credential inAuthorization; it never saw thephm_placeholder, the LiteLLM master key, or the Phantom session token; the request path reached the mock as/v1/chat/completions; the LiteLLM log at default verbosity and the LiteLLM environment never contained the stand-in real credential.
In the daily full CI run (pull requests do not run it): crates/phantom-proxy/tests/litellm_gateway_hermetic_test.rs (cargo test -p phantom-secrets-proxy --test litellm_gateway_hermetic_test). It needs no LiteLLM and no network beyond 127.0.0.1. The test plays the LiteLLM client with only the phm_ placeholder and the phantom exec base URL, against the real proxy built the way phantom exec builds it, with a mock OpenAI-compatible upstream and a second mock that no route names. It checks that:
- the placeholder becomes the real key only at the proxy, only in the configured
Authorizationheader, and only toward the configured upstream; a placeholder copied into the request body is passed through, never resolved; - the second mock receives nothing when the client sets a foreign
Hostheader, sends an absolute-form URL, addsX-Forwarded-Host,ForwardedorX-Original-URL, or when the upstream answers with a redirect to it (the proxy returns the 307 and does not follow it); - proxy error bodies (wrong or missing proxy token, unknown route) and upstream bodies that reflect the key (a 401 error and an SSE event) reach the client without it;
- the key is absent from every proxy log line (TRACE level plus stderr) and from the audit ledger (
PHANTOM_AUDIT=1), which does record the intercepted reflections asproxy.response_leak.
Not tested: any real provider, streaming, tool calls, the phantom exec CLI end to end, the Docker image, Postgres or virtual keys, LiteLLM debug logging (it may print api_base, which contains the Phantom session token), the Responses API, and Claude Code, Codex, Gemini CLI or Cursor talking to LiteLLM.
A loopback recipe for putting LiteLLM in front of a local Ollama server, so agents get one OpenAI-style endpoint for local and hosted models. The local route needs no provider key, so there is nothing for Phantom to inject on it; do not run it under phantom exec with real keys it does not need.
model_list:
- model_name: local-coder
litellm_params:
model: ollama_chat/qwen3.8:27b # ollama_chat sends requests to Ollama's /api/chat
api_base: http://127.0.0.1:11434
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
allow_client_side_credentials: falseLITELLM_MASTER_KEY=sk-<your own random value> \
litellm --config local.yaml --host 127.0.0.1 --port 4000- The
ollama_chat/prefix andapi_baseform come from LiteLLM's Ollama provider docs (checked 2026-10-25). Pin LiteLLM as described in "Pin and verify LiteLLM first". - Keep both Ollama and LiteLLM on
127.0.0.1. Local servers can log prompts, so keepphm_placeholders in prompts. - To mix in a hosted model, add its entry from the Quickstart config and start LiteLLM under
phantom execas shown there. - Not tested: this config was not run against a real LiteLLM or Ollama here.
This section covers the other direction: a Phantom route whose upstream is a LiteLLM proxy reached over HTTPS, with the LiteLLM virtual key in the vault and only its phm_ placeholder in the client.
LiteLLM's docs (checked 2026-10-19) name these headers as places its proxy reads a key from:
- Gateway Auth Reference, "Client to LiteLLM": "
x-litellm-api-key: Bearer sk-...| Preferred LiteLLM Virtual Key header", "Authorization: Bearer sk-...| Standard fallback. Stripped of theBearerprefix before lookup.", and "API-Key,x-api-key,x-goog-api-key,Ocp-Apim-Subscription-Key| Vendor-specific aliases (Azure, Anthropic, Google AI Studio, Azure APIM)." The same page notes that LiteLLM's MCP routes do not parse the vendor aliases. - Virtual Keys, "Pass LiteLLM Key in custom header": "Use this to make LiteLLM proxy look for the virtual key in a custom header instead of the default
"Authorization"header", set withgeneral_settings: litellm_key_header_name: "X-Litellm-Key"and sent asX-Litellm-Key: Bearer $LITELLM_API_KEY. - config.yaml settings: "litellm_key_header_name | str | null (reads Authorization) | If set, allows passing LiteLLM keys as a custom header."
A route sets one of these with header and header_format in .phantom.toml. Use Bearer {secret} for x-litellm-api-key, Authorization and a litellm_key_header_name header, as the docs show. The docs show no value format for the vendor aliases; the test uses the bare key ({secret}), which is an assumption, not a documented LiteLLM format.
[services.litellm]
secret_key = "LITELLM_API_KEY"
secret_type = "api_key"
pattern = "litellm.example.internal"
header = "x-litellm-api-key"
header_format = "Bearer {secret}"For such a route the proxy removes the client's copy of that header, injects the real key in its place, and forwards nothing else it owns. Any echo of the key in the response is scrubbed by value, whatever header or body field LiteLLM puts it in.
Limits:
phantom execrefuses this route and runs only Phantom's exact built-in routes; a self-hosted LiteLLM has no fixed host, so it cannot be one.phantom startserves it from.phantom.toml. Why the difference:phantom startis run by a person on project config they have reviewed.phantom execlaunches an agent, and anything in the repository,.phantom.tomlincluded, may have been written by that agent or by an untrusted change, so accepting custom routes there would widen the attack surface. A test pins exec's refusal (custom_litellm_route_is_refused_by_exec_but_built_for_startincrates/phantom-cli/src/commands/exec.rs).- The route is always
https://. A LiteLLM on plain HTTP loopback is not supported (see "Not supported"). - Only the one configured header is swapped. A
phm_placeholder the client puts in a different header is forwarded as is. LiteLLM then rejects a placeholder, not a real key.
Tested in the daily full CI run by crates/phantom-proxy/tests/litellm_upstream_headers_test.rs (cargo test -p phantom-secrets-proxy --test litellm_upstream_headers_test); pull requests do not run it. For each of the seven headers above it builds the route from a .phantom.toml-style service, points it at a plain-HTTP mock on 127.0.0.1, and checks that the mock gets the real key exactly once in that header, never sees the placeholder or the proxy token, and that echoes of the key (whole or its last half, in a response header and a JSON body) do not reach the client. A second test does the same for a streamed (text/event-stream, chunked) response that echoes the key split across chunks, then its last half, and checks that the stream still completes with data: [DONE]. Not tested: a real LiteLLM and phantom start end to end.
- Cursor with a loopback gateway (see "Connect an agent"). Cursor's CLI is not supported by LiteLLM.
- Phantom in front of LiteLLM (agent to Phantom to LiteLLM) needs a plain-HTTP loopback upstream route, which Phantom does not allow today. See the security review draft before any such change.
When the project has a LiteLLM proxy config (litellm_config.yaml, litellm.yaml or a config.yaml with model_list) whose api_base is a loopback address or which carries a phm_ placeholder, phantom doctor reports:
- a warning for a LiteLLM version below 1.83.0 pinned in
requirements.txt,pyproject.toml, aDockerfileor a compose file (litellm==Xor aberriai/litellmimage tag), - a warning for
allow_client_side_credentials: true, - a warning for an explicit non-loopback bind:
--hostwith a non-loopback address, or a published Docker port without a127.0.0.1host IP, - an info line when no version pin is found.
The check is report-only. It reads the files as text, never runs LiteLLM, and --fix changes nothing. Tests: doctor_json_reports_risky_litellm_through_phantom_and_changes_nothing and doctor_litellm::tests.