Skip to content

Commit 330f21d

Browse files
D-Sai-VenkateshD-Sai-Venkatesh
authored
Pre admission processor (llm-d#1244)
* feat: add PreAdmissionProcessor plugin extension point Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * feat: add agent identity pre-admission processor plugin Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * fix(agentidentity): drop previous_response_id and x-gateway-inference-agent-id Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * refactor: drop reqCtx.FairnessID, derive from headers in director Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * test(director): cover fairness ID derivation from header Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * feat(agentidentity): add Codex session_id, drop OpenCode parent fallback Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): add plugin README Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): fix fairness header name Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * refactor: rename PreAdmissionProcessorExtensionPoint to PreAdmissionExtensionPoint Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * fix(agentidentity): track Codex 0.131.0 hyphenated session-id header Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * feat(agentidentity): configurable additionalSessionHeaders parameter Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * refactor: rename preAdmissionProcessors field/methods to preAdmissionPlugins Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): document additionalSessionHeaders, modernize Codex header, generalize examples Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): use dummy ANTHROPIC_AUTH_TOKEN, clarify it's unused without master_key Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): fix LiteLLM header forwarding config section Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * fix(agentidentity): lowercase priorityHeaders to match upstream header map Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): mention new x-llm-d-inference-fairness-id name Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * rename: PreAdmissionProcessor->PreAdmitter, AdmitRequest->Admit Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * address review comments: Admit blocking note, surface deny reason, default fairness ID after PreAdmitter Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * rename: preadmissionprocessor dir -> preadmitter Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs: rename PreAdmissionProcessor in agent-identity README Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * chore: adapt agent-identity factory to upstream *json.Decoder signature Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * docs(agentidentity): align How It Works with new defaulting flow Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * fix(director): use fmt.Errorf to satisfy perfsprint lint Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> * chore(agentidentity): add compile-time PreAdmitter interface assertion Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com> --------- Signed-off-by: Dasari Surya Sai Venkatesh <suryasai.venkatesh@gmail.com>
1 parent 27122d9 commit 330f21d

17 files changed

Lines changed: 656 additions & 58 deletions

File tree

‎cmd/epp/runner/runner.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ import (
8686
preciseproducer "github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/dataproducer/preciseprefixcache"
8787
latencyproducer "github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/dataproducer/predictedlatency"
8888
"github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer"
89+
"github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/preadmitter/agentidentity"
8990
"github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/requestattributereporter"
9091
testresponsereceived "github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requestcontrol/test/responsereceived"
9192
"github.com/llm-d/llm-d-router/pkg/epp/framework/plugins/requesthandling/parsers/openai"
@@ -586,6 +587,8 @@ func (r *Runner) registerInTreePlugins() {
586587
fwkplugin.Register(utilization.UtilizationDetectorType, utilization.UtilizationDetectorFactory)
587588
// register discovery plugins
588589
fwkplugin.Register(discoveryfile.PluginType, discoveryfile.Factory)
590+
// register pre-admission processor plugins
591+
fwkplugin.Register(agentidentity.PluginType, agentidentity.PluginFactory)
589592
}
590593

591594
func (r *Runner) parseConfigurationPhaseOne(ctx context.Context, opts *runserver.Options) (*configapi.EndpointPickerConfig, error) {

‎pkg/epp/framework/interface/requestcontrol/plugins.go‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import (
2525
)
2626

2727
const (
28+
PreAdmissionExtensionPoint = "PreAdmission"
2829
PreRequestExtensionPoint = "PreRequest"
2930
ResponseReceivedExtensionPoint = "ResponseReceived"
3031
ResponseStreamingExtensionPoint = "ResponseStreaming"
@@ -76,7 +77,14 @@ type DataProducer interface {
7677
// the request is admitted only if all plugins say that the request should be admitted.
7778
type Admitter interface {
7879
plugin.Plugin
79-
// AdmitRequest returns the denial reason, wrapped as error if the request is denied.
80+
// Admit returns the denial reason, wrapped as error if the request is denied.
8081
// If the request is allowed, it returns nil.
81-
AdmitRequest(ctx context.Context, request *fwksched.InferenceRequest, pods []fwksched.Endpoint) error
82+
Admit(ctx context.Context, request *fwksched.InferenceRequest, pods []fwksched.Endpoint) error
83+
}
84+
85+
// PreAdmitter runs after InferenceRequest creation but before admission control.
86+
// It can mutate InferenceRequest fields such as FairnessID and Headers.
87+
type PreAdmitter interface {
88+
plugin.Plugin
89+
PreAdmit(ctx context.Context, request *fwksched.InferenceRequest) error
8290
}

‎pkg/epp/framework/interface/scheduling/types.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ type InferenceRequest struct {
5252
Headers map[string]string
5353
// Request Objective
5454
Objectives RequestObjectives
55+
// FairnessID is the identity used by the flow control system to group requests into fairness queues.
56+
FairnessID string
5557
// RequestSizeBytes is the size of the raw request body in bytes when available.
5658
// Used for token estimation (e.g. inputTokens ≈ RequestSizeBytes/4) without parsing body or calling PlainText().
5759
RequestSizeBytes int

‎pkg/epp/framework/plugins/requestcontrol/admitter/latencyslo/plugin.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,13 +95,13 @@ func (p *LatencyAdmission) Consumes() map[fwkplugin.DataKey]any {
9595
}
9696
}
9797

98-
// AdmitRequest rejects sheddable requests if no endpoint can serve them within SLO.
98+
// Admit rejects sheddable requests if no endpoint can serve them within SLO.
9999
//
100100
// Reject only when ALL of:
101101
// - No endpoint has a valid prediction (all violate SLO)
102102
// - No endpoint is idle (all have running requests)
103103
// - No cold pod exists (predictions are reliable)
104-
func (p *LatencyAdmission) AdmitRequest(ctx context.Context, request *fwksched.InferenceRequest, endpoints []fwksched.Endpoint) error {
104+
func (p *LatencyAdmission) Admit(ctx context.Context, request *fwksched.InferenceRequest, endpoints []fwksched.Endpoint) error {
105105
logger := log.FromContext(ctx)
106106
if request == nil {
107107
return nil

‎pkg/epp/framework/plugins/requestcontrol/admitter/latencyslo/plugin_test.go‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ func makeNonSheddableRequest(ttftSLO, tpotSLO string) *fwksched.InferenceRequest
5959
}
6060
}
6161

62-
func TestAdmitRequest(t *testing.T) {
62+
func TestAdmit(t *testing.T) {
6363
plugin := NewLatencyAdmission(LatencyAdmissionDefaultConfig)
6464

6565
tests := []struct {
@@ -209,9 +209,9 @@ func TestAdmitRequest(t *testing.T) {
209209
if tt.setupFn != nil {
210210
tt.setupFn(tt.endpoints)
211211
}
212-
err := plugin.AdmitRequest(context.Background(), tt.request, tt.endpoints)
212+
err := plugin.Admit(context.Background(), tt.request, tt.endpoints)
213213
if (err != nil) != tt.wantErr {
214-
t.Errorf("AdmitRequest() error = %v, wantErr %v", err, tt.wantErr)
214+
t.Errorf("Admit() error = %v, wantErr %v", err, tt.wantErr)
215215
}
216216
})
217217
}

‎pkg/epp/framework/plugins/requestcontrol/admitter/probabilisticadmitter/plugin.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,9 +120,9 @@ func (p *ProbabilisticAdmitter) TypedName() fwkplugin.TypedName {
120120
return p.typedName
121121
}
122122

123-
// AdmitRequest implements requestcontrol.Admitter.
123+
// Admit implements requestcontrol.Admitter.
124124
// Returns nil to admit, or a ResourceExhausted error to reject.
125-
func (p *ProbabilisticAdmitter) AdmitRequest(_ context.Context, request *fwksched.InferenceRequest, pods []fwksched.Endpoint) error {
125+
func (p *ProbabilisticAdmitter) Admit(_ context.Context, request *fwksched.InferenceRequest, pods []fwksched.Endpoint) error {
126126
if request == nil {
127127
return nil
128128
}

‎pkg/epp/framework/plugins/requestcontrol/admitter/probabilisticadmitter/plugin_test.go‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ func TestProtectedTiersAlwaysAdmitted(t *testing.T) {
132132
req := &fwksched.InferenceRequest{
133133
Objectives: fwksched.RequestObjectives{Priority: priority},
134134
}
135-
if err := admitter.AdmitRequest(context.Background(), req, pods); err != nil {
135+
if err := admitter.Admit(context.Background(), req, pods); err != nil {
136136
t.Errorf("priority %d should always be admitted, got: %v", priority, err)
137137
}
138138
}
@@ -145,7 +145,7 @@ func TestDroppableRejectedAtHighSaturation(t *testing.T) {
145145
req := &fwksched.InferenceRequest{
146146
Objectives: fwksched.RequestObjectives{Priority: -1},
147147
}
148-
if err := admitter.AdmitRequest(context.Background(), req, pods); err == nil {
148+
if err := admitter.Admit(context.Background(), req, pods); err == nil {
149149
t.Error("expected rejection at high saturation")
150150
}
151151
}
@@ -156,7 +156,7 @@ func TestDroppableAdmittedAtZeroSaturation(t *testing.T) {
156156
req := &fwksched.InferenceRequest{
157157
Objectives: fwksched.RequestObjectives{Priority: -1},
158158
}
159-
if err := admitter.AdmitRequest(context.Background(), req, pods); err != nil {
159+
if err := admitter.Admit(context.Background(), req, pods); err != nil {
160160
t.Errorf("expected admission at zero saturation, got: %v", err)
161161
}
162162
}
@@ -171,7 +171,7 @@ func TestNilMetricsTreatedAsSaturated(t *testing.T) {
171171
req := &fwksched.InferenceRequest{
172172
Objectives: fwksched.RequestObjectives{Priority: -1},
173173
}
174-
if err := admitter.AdmitRequest(context.Background(), req, []fwksched.Endpoint{pod}); err == nil {
174+
if err := admitter.Admit(context.Background(), req, []fwksched.Endpoint{pod}); err == nil {
175175
t.Error("expected rejection when pod has nil metrics")
176176
}
177177
}
@@ -181,15 +181,15 @@ func TestNoPods(t *testing.T) {
181181
req := &fwksched.InferenceRequest{
182182
Objectives: fwksched.RequestObjectives{Priority: -1},
183183
}
184-
if err := admitter.AdmitRequest(context.Background(), req, nil); err != nil {
184+
if err := admitter.Admit(context.Background(), req, nil); err != nil {
185185
t.Errorf("no pods should admit, got: %v", err)
186186
}
187187
}
188188

189189
func TestNilRequest(t *testing.T) {
190190
admitter := newAdmitter(defaultParams())
191191
pods := []fwksched.Endpoint{newEndpoint("pod-0", 0, 0)}
192-
if err := admitter.AdmitRequest(context.Background(), nil, pods); err != nil {
192+
if err := admitter.Admit(context.Background(), nil, pods); err != nil {
193193
t.Errorf("nil request should admit, got: %v", err)
194194
}
195195
}
@@ -204,7 +204,7 @@ func TestMultiplePodsSaturationAveraging(t *testing.T) {
204204
req := &fwksched.InferenceRequest{
205205
Objectives: fwksched.RequestObjectives{Priority: -1},
206206
}
207-
if err := admitter.AdmitRequest(context.Background(), req, pods); err == nil {
207+
if err := admitter.Admit(context.Background(), req, pods); err == nil {
208208
t.Error("expected rejection with avg saturation 1.0")
209209
}
210210
}
@@ -220,7 +220,7 @@ func TestQuinticProperty(t *testing.T) {
220220
if expected := math.Min(math.Pow(sat, 5)*300, 1.0); expected < 0.99 {
221221
t.Errorf("expected prob~1.0 at sat=%.4f, got %.4f", sat, expected)
222222
}
223-
if err := admitter.AdmitRequest(context.Background(), req, pods); err == nil {
223+
if err := admitter.Admit(context.Background(), req, pods); err == nil {
224224
t.Error("expected rejection at sat≈0.34 (prob=1.0)")
225225
}
226226
}
@@ -232,7 +232,7 @@ func TestErrorTypeIsStructured(t *testing.T) {
232232
req := &fwksched.InferenceRequest{
233233
Objectives: fwksched.RequestObjectives{Priority: -1},
234234
}
235-
err := admitter.AdmitRequest(context.Background(), req, pods)
235+
err := admitter.Admit(context.Background(), req, pods)
236236
if err == nil {
237237
t.Fatal("expected rejection")
238238
}
@@ -254,13 +254,13 @@ func TestProbabilisticShedding(t *testing.T) {
254254

255255
// rand=0.9 > prob≈0.096 → admit
256256
if err := newAdmitter(defaultParams()).WithRandFn(func() float64 { return 0.9 }).
257-
AdmitRequest(context.Background(), req, pods); err != nil {
257+
Admit(context.Background(), req, pods); err != nil {
258258
t.Errorf("expected admission with rand=0.9 > prob≈0.096, got: %v", err)
259259
}
260260

261261
// rand=0.0 < prob≈0.096 → reject
262262
if err := newAdmitter(defaultParams()).WithRandFn(func() float64 { return 0.0 }).
263-
AdmitRequest(context.Background(), req, pods); err == nil {
263+
Admit(context.Background(), req, pods); err == nil {
264264
t.Error("expected rejection with rand=0.0 < prob≈0.096")
265265
}
266266
}

‎pkg/epp/framework/plugins/requestcontrol/preadmitter/agentidentity/README.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Agent Identity
2+
3+
**Type:** `agent-identity`
4+
**Interfaces:** `requestcontrol.PreAdmitter`
5+
6+
Resolves a per-session identity from agent-specific HTTP headers and writes it into `InferenceRequest.FairnessID`, so every turn of an agent session lands in the same flow-control fairness queue.
7+
8+
## What It Does
9+
10+
The plugin runs after request assembly and before admission control. If the request does not already carry an explicit fairness ID (`x-llm-d-inference-fairness-id`, or its deprecated alias `x-gateway-inference-fairness-id`), it inspects a fixed set of agent session headers and copies the first non-empty value into `FairnessID`. The flow-control layer keys its queues on `FlowKey{ID: FairnessID, Priority}`, so this turns "all turns from one agent session" into "all turns share one queue."
11+
12+
Without it, every request from a given agent session falls into the default fairness queue alongside unrelated traffic, and per-session fairness, prefix-cache affinity, and per-tenant rate limiting all collapse to per-request granularity.
13+
14+
## How It Works
15+
16+
1. If `request.FairnessID` is already non-empty, return immediately — an explicit upstream `x-llm-d-inference-fairness-id` (or its deprecated alias `x-gateway-inference-fairness-id`) is read into `FairnessID` before the plugin runs and always wins over a derived one.
17+
2. Otherwise, walk the priority list of agent session headers and copy the first non-empty match into `request.FairnessID`. Operator-supplied entries from `additionalSessionHeaders` come first, followed by the built-in defaults in this order:
18+
1. `x-claude-code-session-id` (Claude Code)
19+
2. `x-session-affinity` (OpenCode)
20+
3. `session-id` (Codex)
21+
4. `session_id` (Codex, legacy underscored fallback)
22+
3. If nothing matches, leave `FairnessID` empty and return — the director applies `metadata.DefaultFairnessID` after the plugin returns, so the request is still admitted, just into the shared default queue.
23+
24+
The plugin is stateless and safe under concurrent use.
25+
26+
## Inputs Consumed
27+
28+
- `scheduling.InferenceRequest.Headers` — read-only lookup of the session headers above (built-in defaults plus any from `additionalSessionHeaders`). Keys are expected lowercase (Envoy normalizes inbound headers).
29+
- `scheduling.InferenceRequest.FairnessID` — read to detect an upstream override; written when an agent header matches.
30+
31+
## Configuration
32+
33+
**Location:** Top-level `plugins:` list in the `EndpointPickerConfig`.
34+
**Enabled by default:** No. Add a `- type: agent-identity` entry to enable; the runner discovers it as a `PreAdmitter` and wires it in.
35+
36+
### Parameters
37+
38+
| Name | Type | Required | Default | Description |
39+
|------|------|----------|---------|-------------|
40+
| `additionalSessionHeaders` | `[]string` | No | `[]` | Extra header names to check before the built-in defaults. Order is preserved; the first non-empty match wins. Use this to support a new agent, or to track an upstream rename, without a code change. |
41+
42+
### Examples
43+
44+
Default configuration — no parameters, only the built-in headers are checked:
45+
46+
```yaml
47+
apiVersion: inference.networking.x-k8s.io/v1alpha1
48+
kind: EndpointPickerConfig
49+
plugins:
50+
- type: agent-identity
51+
```
52+
53+
With additional headers — checked before the built-in defaults (header names are arbitrary; substitute whatever the agent actually emits):
54+
55+
```yaml
56+
apiVersion: inference.networking.x-k8s.io/v1alpha1
57+
kind: EndpointPickerConfig
58+
plugins:
59+
- type: agent-identity
60+
parameters:
61+
additionalSessionHeaders:
62+
- x-my-agent-session
63+
- x-another-agent-id
64+
```
65+
66+
### Per-agent client setup
67+
68+
The plugin only reads headers — getting them onto the wire is the agent's job. Each supported agent has different requirements.
69+
70+
#### Claude Code — **LiteLLM is required**
71+
72+
Claude Code speaks Anthropic's Messages API. llm-d's gateway exposes the OpenAI chat-completions wire format, so a translator is required in the path. LiteLLM works:
73+
74+
```yaml
75+
# LiteLLM proxy config (pass to `litellm --config <path>`)
76+
model_list:
77+
- model_name: <client-facing-model-name>
78+
litellm_params:
79+
model: hosted_vllm/<upstream-model-name>
80+
api_base: http://<llmd-gateway>/v1
81+
82+
general_settings:
83+
forward_client_headers_to_llm_api: true
84+
```
85+
86+
`forward_client_headers_to_llm_api: true` is **required** — without it LiteLLM strips `x-claude-code-session-id` (and every other `x-*` header) on the way to the upstream, and the plugin sees nothing.
87+
88+
Then point Claude Code at LiteLLM and launch it. Use a settings file (rather than env vars) so inherited user-level settings, OAuth credentials, or keychain entries cannot override the configuration:
89+
90+
```json
91+
// Claude Code settings file (any path, e.g. /tmp/claude-llmd-settings.json)
92+
{
93+
"env": {
94+
"ANTHROPIC_BASE_URL": "http://<litellm-host>",
95+
"ANTHROPIC_AUTH_TOKEN": "dummy",
96+
"ANTHROPIC_MODEL": "<client-facing-model-name>"
97+
}
98+
}
99+
```
100+
101+
```bash
102+
claude --bare --settings <path-to-settings.json> --setting-sources ""
103+
```
104+
105+
`--bare` disables OAuth and keychain reads; `--setting-sources ""` disables loading any other settings file. Together they ensure only the file passed via `--settings` is used.
106+
107+
`<client-facing-model-name>` must match the `model_name` declared in the LiteLLM `model_list` above. `ANTHROPIC_AUTH_TOKEN` is required by Claude Code but its value is unused when LiteLLM has no `master_key` set — any non-empty string works. Claude Code emits `x-claude-code-session-id` automatically on every outbound request — no further client config needed.
108+
109+
#### OpenCode — **No LiteLLM required**
110+
111+
OpenCode uses Vercel's AI SDK with `@ai-sdk/openai-compatible` and speaks OpenAI chat-completions natively, so it talks to the llm-d gateway directly.
112+
113+
```json
114+
// ~/.config/opencode/opencode.json
115+
{
116+
"$schema": "https://opencode.ai/config.json",
117+
"provider": {
118+
"llmd-local": {
119+
"npm": "@ai-sdk/openai-compatible",
120+
"name": "llmd-local",
121+
"options": {
122+
"baseURL": "http://<llmd-gateway>/v1",
123+
"apiKey": "dummy"
124+
},
125+
"models": {
126+
"<upstream-model-name>": { "name": "<display-name>" }
127+
}
128+
}
129+
}
130+
}
131+
```
132+
133+
OpenCode emits `x-session-affinity` automatically on every outbound request.
134+
135+
#### Codex — **No LiteLLM required**
136+
137+
Codex emits a session header automatically on every outbound request. Current builds use the hyphenated `session-id` (no `x-` prefix); older builds use the underscored `session_id` form, which the plugin still recognizes as a fallback.
138+
139+
## Limitations
140+
141+
- **Default-queue fall-through is silent.** Requests from agents that don't match any of the configured headers land in the default fairness queue without any indication. This is by design (the plugin is non-fatal), but operators should not assume the absence of errors means every client is being identified.
142+
- **Codex `previous_response_id` is not used.** It references the prior turn's response, not the chain root, so keying on it would shard one conversation across many queues. Correctly folding it back to the root requires a `ResponseBody` hook recording `response.id → root` mappings, which this plugin does not implement.
143+
144+
## Related Documentation
145+
- Claude Code session header (official): <https://code.claude.com/docs/en/llm-gateway> — the `X-Claude-Code-Session-Id` row in "Request headers Claude Code includes."
146+
- OpenCode session header (Cloudflare announcement, documents the `x-session-affinity` contract): <https://blog.cloudflare.com/workers-ai-large-models/>
147+
- Codex session header (Codex CLI source — `build_session_headers` inserts `session_id` as an HTTP header on every outbound request; OpenAI does not document this in the public docs): <https://github.com/openai/codex/blob/d2e18246c96e8b440f9d97135356d37f3f3b4d63/codex-rs/codex-api/src/requests/headers.rs>

0 commit comments

Comments
 (0)