TruLens dashboard not showing token usage and cost with Ollama provider

[RockybhaiRakesh]

Bug Description
When using TruLens + Ollama (local LLM via LiteLLM provider), the TruLens dashboard always shows 0 for Total Tokens (#) and Total Cost (App), even though queries and responses are being logged successfully.

Latency is shown correctly, but token usage and cost remain 0.

To Reproduce
Steps to reproduce the behavior:

Install and run Ollama locally (ollama run llama2, ollama run nomic-embed-text).

Install dependencies:

pip install trulens trulens-apps-langchain trulens-providers-litellm langchain-community chromadb requests

Run the following minimal script:

import requests, chromadb
from langchain_community.llms import Ollama
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from trulens.core import Feedback, TruSession
from trulens.apps.langchain import TruChain
from trulens.providers.litellm import LiteLLM

ollama = Ollama(base_url="http://localhost:11434", model="llama2")

full_prompt = HumanMessagePromptTemplate(
prompt=PromptTemplate(
template="Context: {context}\n\nQuestion: {prompt}\nAnswer:",
input_variables=["context", "prompt"],
)
)
chat_prompt_template = ChatPromptTemplate.from_messages([full_prompt])
chain = LLMChain(llm=ollama, prompt=chat_prompt_template, verbose=True)

session = TruSession()
session.reset_database()

ollama_provider = LiteLLM(model_engine="ollama/llama2", api_base="http://localhost:11434")
relevance = Feedback(ollama_provider.relevance_with_cot_reasons).on_input_output()

tru_recorder = TruChain(chain, app_name="Llama2_Chroma_RAG", feedbacks=[relevance])

with tru_recorder as recording:
response = chain.invoke({"context": "Ollama is a local LLM runtime.", "prompt": "What is Ollama?"})
print(response["text"])

from trulens.dashboard import run_dashboard
run_dashboard(session)

Open the dashboard (http://localhost:8501).

Expected Behavior
The dashboard should display Total Tokens (#) and ideally allow either:

integration with Ollama’s internal token counters (if available), or

a clear warning that token/cost metrics are not supported for local models.

Relevant Logs/Tracebacks
No error logs. Application runs successfully.
Dashboard shows 0 for tokens and cost:

Environment:

OS: Windows 11

Python Version: 3.12.4

TruLens Version: (e.g. 2.3.0)

LangChain Version: 0.2.x

LiteLLM Version: 1.11.1

Ollama Version: latest (running ollama serve)

Models Installed:

ollama list
llama2:latest
nomic-embed-text:latest

[sfc-gh-nvytla]

@RockybhaiRakesh, thanks for reporting this. One thing to clarify: the screenshot you attached is not of the TruLens dashboard, but of a Google Cloud console. Could you please add a screenshot of the issue you're seeing on the TruLens dashboard?

[RockybhaiRakesh]

Hello @sfc-gh-nvytla,

Thanks for pointing that out. You’re right — the screenshot I attached earlier was from Google Cloud console by mistake.

Here is the TruLens dashboard screenshot showing the issue (Total Tokens = 0 and Total Cost = 0, even though queries/responses are logged and latency is displayed correctly):

my git hub link - https://github.com/RockybhaiRakesh/trulens_ollama

sample code is below

import requests
import json
import chromadb

from langchain_community.llms import Ollama

from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.prompts.chat import ChatPromptTemplate, HumanMessagePromptTemplate

from trulens.core import Feedback, TruSession
from trulens.apps.langchain import TruChain
from trulens.providers.litellm import LiteLLM

---------------- Ollama + Chroma Setup ----------------

OLLAMA_GENERATE_URL = "http://localhost:11434/api/generate"
OLLAMA_EMBED_URL = "http://localhost:11434/api/embed"

CHROMA_API_KEY = "ck-"
CHROMA_TENANT = ""
CHROMA_DATABASE = "demo1"

chroma_client = chromadb.CloudClient(
api_key=CHROMA_API_KEY,
tenant=CHROMA_TENANT,
database=CHROMA_DATABASE
)

collection = chroma_client.get_or_create_collection(name="demo_collection")

---------------- Embedding Function ----------------

def get_embedding(text: str):
response = requests.post(
OLLAMA_EMBED_URL,
json={"model": "nomic-embed-text", "input": text}
)
data = response.json()
return data["embeddings"][0]

---------------- Add Documents ----------------

documents = {
"uw_info": "The University of Washington, founded in 1861 in Seattle, is a public research university with over 45,000 students.",
"wsu_info": "Washington State University, founded in 1890, is a public research university in Pullman, Washington.",
"starbucks_info": "Starbucks is an American multinational coffeehouse chain headquartered in Seattle, Washington."
}

for doc_id, text in documents.items():
emb = get_embedding(text)
collection.add(documents=[text], ids=[doc_id], embeddings=[emb])

print("✅ Documents stored in Chroma Cloud")

---------------- LangChain + Ollama ----------------

ollama = Ollama(base_url="http://localhost:11434", model="llama2")

full_prompt = HumanMessagePromptTemplate(
prompt=PromptTemplate(
template="Context: {context}\n\nQuestion: {prompt}\nAnswer:",
input_variables=["context", "prompt"],
)
)
chat_prompt_template = ChatPromptTemplate.from_messages([full_prompt])
chain = LLMChain(llm=ollama, prompt=chat_prompt_template, verbose=True)

---------------- TruLens Setup ----------------

session = TruSession()
session.reset_database()

ollama_provider = LiteLLM(
model_engine="ollama/llama2",
api_base="http://localhost:11434"
)

relevance = Feedback(
ollama_provider.relevance_with_cot_reasons
).on_input_output()

tru_recorder = TruChain(
chain, app_name="Llama2_Chroma_RAG", feedbacks=[relevance]
)

---------------- Query Function ----------------

def rag_query(query: str):
q_emb = get_embedding(query)
results = collection.query(query_embeddings=[q_emb], n_results=2)
context = " ".join([doc for docs in results["documents"] for doc in docs])
with tru_recorder as recording:
response = chain.invoke({"context": context, "prompt": query})
return response["text"]

---------------- Run Example ----------------

if name == "main":
queries = [
"What is the University of Washington?",
"Where is Starbucks headquartered?",
"Tell me about Washington State University."
]

for q in queries:
    print(f"\n Query: {q}")
    answer = rag_query(q)
    print(f" Llama2: {answer}")

# ---------------- Run TruLens Dashboard ----------------
from trulens.dashboard import run_dashboard
run_dashboard(session)