Is it possible to log the final prompts sent to the LLM? Similar to langchain's verbose=True

[ro0sterjam]

It would really be helpful for debugging (and learning) to be able to understand what is happening under the hood for functions such as createChatCompletion. Is there a way to do this?

[luebken]

+1

Bumping this old message. What is the expected way to debug the underlying LLM calls?

[luebken]

It can be very well the case that's not an api error:

    onError(errorObj) {
      console.error("=== Error Object ===");
      console.error("Type:", Object.prototype.toString.call(errorObj));
      console.error("Constructor:", errorObj?.constructor?.name);
      console.error("Instance of Error:", errorObj instanceof Error);
      console.error("Keys:", Object.keys(errorObj));
      console.error("Full object:", JSON.stringify(errorObj, null, 2));

results in

=== Error Object ===
Type: [object Object]
Constructor: Object
Instance of Error: false
Keys: [ 'error' ]
Full object: {
  "error": {
    "type": "error",
    "sequence_number": 2,
    "error": {
      "type": "invalid_request_error",
      "code": "context_length_exceeded",
      "message": "Your input exceeds the context window of this model. Please adjust your input and try again.",
      "param": "input"
    }
  }
}

[lgrammel]

Huh. Does it have a trace to where it is thrown?

[luebken]

No. I just get this error object. I'm currently hypothesizing some problem with my messages and that the error might be even out of the AI SDK itself. But I can't find any hint on how to turn up debug log verbosity. Apologies for the novice nextjs questions here.

[luebken]

I found the root-cause. I had a false token count due to silent fallback due to broken wasm support with nextjs/turbopack.

As for the missing trace: switching from streamtext to generatetext got me access to the request

[luebken]

I can think of the following improvements for the sdk:

• Document the above in some debug section in the docs
• Built in support for tokenizer

@lgrammel if you feel worthwhile I'm happy to create issues.

Thanks for the help.