Errors from extproc not returned to user causing user timeouts

[sukumargaonkar]

Description:

What issue is being seen? Describe what should be happening instead of
the bug, for example: Envoy should not crash, the expected value isn't
returned, etc.

If an error is returned during extproc's request/response header/body processing, the error is not returned to the user. This causes user to keep waiting for a response until it eventually timesout

expected
an error message should be immediately returned to the user.

Repro steps:

Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Explicitly return a fake error in any of the extproc's processing methods (eg. requestBodyProcessing) and send a request to it.

Note: If there are privacy concerns, sanitize the data prior to
sharing.

Environment:

Include the environment like gateway version, envoy version and so on.

Logs:

Include the access logs and the Envoy logs.

Example logs.

time=2025-07-08T11:16:40.329-04:00 level=ERROR msg="error processing request message" error="cannot process request headers: failed to transform request: fake Error"

Possible Cause
Error returned from processMsg() here is never streamed back to user

Instead ProcessingResponse_ImmediateResponse should be returned to the user

[yanavlasov]

It is controlled by the fail-close configuration in ext_proc filter. I think we need to configured ext_proc as fail-close since it is required for gateway to work.

[yuzisun]

example timeout instead of validation failure when you send a message to GCPAnthropic without setting the max token size

'{"model": "gcp.claude-3-7-sonnet", "messages": [{"role": "user", "content": "What is capital of France?"}], "stream": false}'

time=2025-08-12T09:20:34.490Z level=ERROR msg="error processing request message" error="cannot process request 
headers: failed to transform request: the maximum number of tokens must be set for Anthropic, got nil instead"

[mathetake]

--- a/tests/extproc/testupstream_test.go
+++ b/tests/extproc/testupstream_test.go
@@ -201,7 +201,7 @@ func TestWithTestUpstream(t *testing.T) {
                        path:            "/v1/chat/completions",
                        method:          http.MethodPost,
                        requestBody:     `{"model":"claude-3-sonnet","max_completion_tokens":1024, "messages":[{"role":"system","content":"You are an Anthropic assistant."},{"role":"user","content":"Hello!"}]}`,
-                       expRequestBody:  `{"max_tokens":1024,"messages":[{"content":[{"text":"Hello!","type":"text"}],"role":"user"}],"system":[{"text":"You are an Anthropic assistant.","type":"text"}],"anthropic_version":"vertex-2023-10-16"}`,
+                       expRequestBody:  `{"messages":[{"content":[{"text":"Hello!","type":"text"}],"role":"user"}],"system":[{"text":"You are an Anthropic assistant.","type":"text"}],"anthropic_version":"vertex-2023-10-16"}`,
                        expHost:         "gcp-region-aiplatform.googleapis.com",
                        expPath:         "/v1/projects/gcp-project-name/locations/gcp-region/publishers/anthropic/models/claude-3-sonnet:rawPredict",
                        expHeaders:      map[string]string{"Authorization": "Bearer " + fakeGCPAuthToken},

This doesn't timeout at least extproc level... let me take a look at e2e test

[mathetake]

ah i got the repro

[mathetake]

I found a root cause. The ext_proc implementation on Envoy side ignores immediate response when configured as an upstream filter, so any error happening there (e.g. translation) results in the timeout. I am not sure if there's any workaround as it's unconditional logic for upstream filters as I see here

https://github.com/envoyproxy/envoy/blob/150411af676af070dca4c6d60ec37e57d4fc672d/source/extensions/filters/http/ext_proc/ext_proc.cc#L1730-L1734

[mathetake]

envoyproxy/envoy#33273 (comment)

[mathetake]

At least this seems like not a limitation of Envoy core but ext_proc implementation matter (or it's async nature complicates things. Dynamic modules can send an immediate request from an upstream filter.

[mathetake]

so this is not actionable stuff for v0.3. The workaround is to set a proper timeout with ClientTrafficPolicy at the moment until either we add support for dynamic modules implementation or fix the envoy (which can be pat of 1.36).