generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 218
Open
Labels
needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.
Description
What would you like to be added:
I propose updating the ResponseComplete plugin interface signature to include an error parameter (or a structured status object).
Currently, the signature is:
ResponseComplete(ctx context.Context, request *types.LLMRequest, response *Response, targetPod *backend.Pod)It should be updated to something resembling:
ResponseComplete(..., err error)
// OR
ResponseComplete(..., status TerminationStatus)
// OR
ResponseComplete(..., reason string)Why is this needed:
With the recent merge of PR #2064, the ResponseComplete hook is now guaranteed to execute symmetrically with PreRequest, covering all termination states:
- Successful completion.
- Internal errors (e.g., JSON marshaling failures).
- Client disconnects / Context cancellation.
The Problem:
Currently, a plugin implementing this hook has no way to distinguish between a successful request and a cancelled/failed one. This limitation prevents plugins from accurately recording observability data (e.g., incrementing an error_count metric vs. a success_count, or logging the cause of a disconnect).
Notes:
- This is a breaking change for existing plugins.
Metadata
Metadata
Assignees
Labels
needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.