Skip to content

Proposal: Pass error context to ResponseComplete extension point #2079

@LukeAVanDrie

Description

@LukeAVanDrie

What would you like to be added:

I propose updating the ResponseComplete plugin interface signature to include an error parameter (or a structured status object).

Currently, the signature is:

ResponseComplete(ctx context.Context, request *types.LLMRequest, response *Response, targetPod *backend.Pod)

It should be updated to something resembling:

ResponseComplete(..., err error)
// OR
ResponseComplete(..., status TerminationStatus)
// OR
ResponseComplete(..., reason string)

Why is this needed:

With the recent merge of PR #2064, the ResponseComplete hook is now guaranteed to execute symmetrically with PreRequest, covering all termination states:

  1. Successful completion.
  2. Internal errors (e.g., JSON marshaling failures).
  3. Client disconnects / Context cancellation.

The Problem:
Currently, a plugin implementing this hook has no way to distinguish between a successful request and a cancelled/failed one. This limitation prevents plugins from accurately recording observability data (e.g., incrementing an error_count metric vs. a success_count, or logging the cause of a disconnect).

Notes:

  • This is a breaking change for existing plugins.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions