How to Get Token Usage (Prompt & Completion) When Using Streaming Response in Prism PHP?

[hellodit]

Hi, I’m currently working with streaming responses using OpenAI via Prism PHP, and I’d like to ask about tracking token usage (specifically prompt tokens and completion tokens) when using the asStream() method.

Here’s a simplified version of the code I’m using:

// if agent has tools, add them to the system prompt
$prismTools = [];
if ($this->agent->tools->count() > 0) {
    $tools = $this->agent->tools->where('status', Tool::STATUS_ACTIVE);
    foreach ($tools as $tool) {
        $prismTools[] = new GenericVectorSearchTool(
            $tool->code,
            $tool->description,
            $this->agent
        );
    }
}

// streaming the response
$stream = Prism::text()
    ->using(Provider::OpenAI, $this->agent->chat_model)
    ->withMaxSteps(5) // Increased to allow for multiple tool calls
    ->withTools($prismTools)
    ->withMessages($messages)
    ->asStream();

foreach ($stream as $chunk) {
    $onChunk($chunk->text);
}

My question is:
How can I access or calculate the number of prompt tokens and completion tokens used in this streaming context?
In a non-streaming response, I can usually access this via the usage property from the API response. But when using streaming mode like in the example above, I’m not sure how or where to retrieve that information.

Any suggestions or best practices for tracking token usage in this case would be really helpful. Thanks in advance!

[hellodit]

I finally found the answer. Since there's no built-in function to get the token count, we need to calculate it manually. Fortunately, I came across a package called yethee/tiktoken-php that helps with the token calculation.

I’m not sure about its performance yet, but for now, it solves my problem.

Here’s a helper function I created to get the token count:

public static function countTokens(string $model, string $text): int
    {
        // Inisialisasi provider dan encoder sesuai model
        $provider = new EncoderProvider();
        $encoder = $provider->getForModel($model);

        // Hitung token prompt
        $totalToken = count($encoder->encode($text));
        return $totalToken;
    }

[b-barry]

@hellodit I did not understand how do you linked it with the stream? Could you share more info?

[hellodit]

Imagine you have a function like this:

$finalResponse = null;

$stream = Prism::text()
    ->using(Provider::OpenAI, '4o')
    ->withMaxSteps(5) // Increased steps to allow multiple tool calls
    ->withTools($prismTools)
    ->withMessages($messages)
    ->asStream();

foreach ($stream as $chunk) {
    $finalResponse .= $chunk->text;
    // You can add streaming logic here, like updating UI in real-time
}

Once the streaming is complete, the $finalResponse variable will contain the full response from the AI. If you want to calculate how many tokens it used, you can simply do:

MessageHelper::countTokens(
    model: $agent->chat_model,
    text: $finalResponse
);

So the flow is: stream → collect all chunks into $finalResponse → calculate tokens after it's done. Clean and flexible 🔥