Handle exception for token cost calculation

[emerzon]

Description

The code for token cost calculation fails in some cases when using a LiteLLM proxy due to mismatch with the provider+model naming. For now, just handle this exception and assume cost 0 when that happens instead of breaking the flow - A more precise, LiteLLM proxy based cost calculation (relying in the /model/info) LiteLLM Proxy implementation will be needed for accurate cost tracking.

How Has This Been Tested?

Test enabling Contextual RAG LLM when a non-standard model name is used in the LLM connector (Ie. claude-3.5-haiku@20241022)

[vercel]

@emerzon is attempting to deploy a commit to the Danswer Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

PR Summary

The PR updates token cost calculation to handle exceptions due to LiteLLM proxy naming mismatches without breaking the flow.

• Modified backend/onyx/llm/utils.py to catch exceptions in get_llm_contextual_cost and log detailed context.
• Returns a fallback cost of 0 when an exception occurs, ensuring uninterrupted processing.
• This temporary measure will later be replaced by a more precise cost calculation implementation.

_{💡 (5/5) You can turn off certain types of comments like style here!}

_{1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile}