[Bug]: How to make stream cancellation provide usage chunk #194
Description
Hello! Please help me.
I am using the fetch API to generate a stream response for conversations we have in our site. We have streaming on and usage on. We would like to be able to cancel the stream at any moment but all the cancel logic on the documentation suggests cutting off the stream which we will NOT be able to get the usage chunk. We need that usage chunk. Am I missing something? Or is the API missing the logic to still return the usage chunk even after a stream cancellation?
The main core of the request is here showing that I am requesting for streaming, usage, and providing an abort controller.
const requestBody = {
model,
messages,
stream: true,
stream_options: {
include_usage: true,
}
};
// Create AbortController for OpenRouter request cancellation
const openRouterAbortController = new AbortController();
const fetchOptions = {
method: "POST",
headers,
body: JSON.stringify(requestBody),
signal: openRouterAbortController.signal,
};
let response;
try {
response = await fetch(openRouterUrl, fetchOptions);
} catch (fetchError) {
...
}The expected result would be to call the abort controller and the stream of chunks ending by including a final finish reason chunk, usage chunk, and that [DONE] message.
To clarify: I am NOT trying to hard-terminate the HTTP request itself.
I need a way to signal the model to stop generation early while still allowing the stream to complete normally so that the final usage chunk can be delivered.
Right now the only cancellation mechanism exposed in documentation is AbortController, which closes the connection entirely and prevents the usage chunk from arriving.
Is there:
- A server-side "stop generation but finish stream" mechanism?
- Or a request_id-based usage retrieval endpoint after cancellation?
If neither exists, this may be a missing API capability rather than a client bug.