Skip to content

Clarification on input tokens #334

Description

@Specy

I'm using the vercel AI sdk and wanted to calculate the pricing of each turn but i get error:

Error: Uncached text input tokens cannot be negative

My code:

export function calculateAiUsage(model: LLMModels, usage: LanguageModelUsage) {
    console.log(usage)
    console.log({
        input_tokens: usage.inputTokenDetails.noCacheTokens,
        cache_read_tokens: usage.inputTokenDetails.cacheReadTokens,
        cache_write_tokens: usage.inputTokenDetails.cacheWriteTokens,
        output_tokens: usage.outputTokens,
    });
    try{
        const price = calcPrice(
            {
                input_tokens: usage.inputTokenDetails.noCacheTokens,
                cache_read_tokens: usage.inputTokenDetails.cacheReadTokens,
                cache_write_tokens: usage.inputTokenDetails.cacheWriteTokens,
                output_tokens: usage.outputTokens,
            },
            LLM_MODELS_TO_NAME[model],
        );

        return price
            ? {
                  total: price.total_price,
                  input: price.input_price,
                  output: price.output_price,
              }
            : null;
    }catch (e){
        console.error(e);
        return null
    }
}

This is the result:

{
  inputTokens: 4740,
  inputTokenDetails: { noCacheTokens: 5, cacheReadTokens: 0, cacheWriteTokens: 4735 },
  outputTokens: 255,
  outputTokenDetails: { textTokens: undefined, reasoningTokens: undefined },
  totalTokens: 4995,
  raw: {
    input_tokens: 5,
    cache_creation_input_tokens: 4735,
    cache_read_input_tokens: 0,
    cache_creation: { ephemeral_5m_input_tokens: 4735, ephemeral_1h_input_tokens: 0 },
    output_tokens: 255,
    service_tier: 'standard',
    inference_geo: 'not_available'
  },
  reasoningTokens: undefined,
  cachedInputTokens: 0
}
{
  input_tokens: 5,
  cache_read_tokens: 0,
  cache_write_tokens: 4735,
  output_tokens: 255
}

I'm confused as of how i'm supposed to map the input tokens to what the library expects, is it supposed to be:

        const price = calcPrice(
            {
                //ALL tokens
                input_tokens: usage.inputToken, 
                //out of which, those were cache read
                cache_read_tokens: usage.inputTokenDetails.cacheReadTokens,
                //out of which those were cache writes
                cache_write_tokens: usage.inputTokenDetails.cacheWriteTokens,
                output_tokens: usage.outputTokens,
            },
            LLM_MODELS_TO_NAME[model],
        );

is input_tokens supposed to be the total number of input tokens (regardless if they were uncached, cache read or written) or only tokens that were not cache read or writes?

EDIT:
looking at the code of the library, yes it seems like input_tokens is supposed to be the total and then cache_read_tokens and cache_write_tokens are partitions of the input_tokens. Maybe it could be made more clear on the docs?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions