TruLens v2.1.0

TruLens v2.1

TruLens 2.1 includes a number of new features and bug fixes to support tracing and evaluation of agents including Inline evals, trajectory evals, native LangGraph instrumentation (via TruGraph). Additionally, we made a variety of stability improvements to evaluators benefiting both OSS and Snowflake users including structured output support and shifting to new more stable serverside metric computation in Snowflake.

New Features

• Create @inline_evaluations decorator. by @sfc-gh-dkurokawa in #2127
• Allow for trace level metrics. by @sfc-gh-dkurokawa in #2075
• Add structured outputs to feedback providers by @sfc-gh-chu in #2098
• Validate openai models support for structured output by @sfc-gh-chu in #2102
• Enable OTel event queries with app_version by @sfc-gh-nvytla in #2099
• Implement TruGraph for Langgraph instrumentation by @sfc-gh-dhuang in #2114
• Support externalbrowser authentication in Snowflake for the Streamlit UI. by @sfc-gh-dkurokawa in #2130
• Create trulens.apps.langgraph package. by @sfc-gh-dkurokawa in #2119
• Minor evaluator improvements. by @sfc-gh-dkurokawa in #2123
• Trajectory Evals (experimental) by @sfc-gh-ajia in #2108
• Run feedback function on everything in event table initially. by @sfc-gh-dkurokawa in #2118
• [SDK][Major Version Change] SDK (Snowflake) uses LLM orchestration layer by @sfc-gh-dhuang in #1969

Bug Fixes

• Feat: Modernize trajectory evals to v2 feedback system with few-shot example support by @sfc-gh-nvytla in #2121
• Fix issues with using JSON in Snowflake regular tables. by @sfc-gh-dkurokawa in #2095
• [Snowflake] [major version release] Remove deprecated SDK orchestration code. by @sfc-gh-dhuang in #2091
• Allow ignoring None values. by @sfc-gh-dkurokawa in #2096
• Fix optionals for criteria ranges by @sfc-gh-chu in #2097
• Update snowflake-connector-python version dependency as the old one doesn't work for some auth stuff. by @sfc-gh-dkurokawa in #2125
• Accept Anaconda's TOS. by @sfc-gh-dkurokawa in #2128
• Fix: Handle division by zero errors in feedback, groundtruth, and hotspots, improve test coverage for groundtruth by @sfc-gh-nvytla in #2124
• Fix conda build issues by restricting smmap version. by @sfc-gh-dkurokawa in #2133
• Test: Add test coverage for GroundTruthAgreement by @sfc-gh-nvytla in #2132
• Handle when the spans don't have app ids in the UI. by @sfc-gh-dkurokawa in #2131
• update meta version by @sfc-gh-chu in #2074
• [Snowflake] Add test dataset notebook for LLM GA work with orchestration layer by @sfc-gh-dhuang in #2086
• [Snowflake][Major version change] Use new sprocs to ensure correct batch ingestion behavior by @sfc-gh-dhuang in #2090
• set alembic path_separator by @sfc-gh-chu in #2106
• Use pydantic ConfigDict and inherit by @sfc-gh-chu in #2105
• Allow ignore_none_values to be set for Selector and set to False by default. by @sfc-gh-dkurokawa in #2111

Docs

• [Docs] Nit: Update grammar for guardrails doc by @sfc-gh-nvytla in #2077
• [Docs] Improve and standardize language for instrumentation-specific docs, fix grammar and spelling errors by @sfc-gh-nvytla in #2078
• [Docs] Fix code examples, update grammar, spelling for logging-specific docs by @sfc-gh-nvytla in #2079
• [Docs] Fix grammatical and spelling errors and standardize language for evaluation-specific component guides and notebooks by @sfc-gh-nvytla in #2072
• remove online link for grit migration by @sfc-gh-jreini in #2085
• Add docs for inline evaluations by @sfc-gh-jreini in #2129
• Add DeepWiki badge to README by @sfc-gh-nvytla in #2088
• Feat: Update and standardize docstrings for llm_provider.py by @sfc-gh-nvytla in #2117
• Docs: Update docs website year, fix homepage link by @sfc-gh-nvytla in #2122
• More updates to selecting span components guides by @sfc-gh-jreini in #2089
• trace level selection docs by @sfc-gh-jreini in #2103
• Update trulens.org homepage with agent & otel copy by @sfc-gh-jreini in #2081
• [Docs] Standardize language, improve formatting for instrumentation, contributing, other docs by @sfc-gh-nvytla in #2080
• Update instrumentation component guides for OTEL by @sfc-gh-jreini in #2082
• remove outdated eval benchmarks from nav by @sfc-gh-jreini in #2084
• update selecting components page for OTel by @sfc-gh-jreini in #2083

Examples

• [Tutorials] Updates for LangChain notebooks by @sfc-gh-nvytla in #2092
• Notebook OTEL conversions and cleanup by @sfc-gh-jreini in #2107
• [Experimental] Add WIP notebook to directly compute feedbacks on OTel spans by @sfc-gh-nvytla in #2100
• Update snowflake data agent/langgraph example by @sfc-gh-jreini in #2120
• Formatting for snowflaketools nb by @sfc-gh-jreini in #2126
• Get more quickstarts OTEL-ready by @sfc-gh-jreini in #2076
• Update example notebooks to use the new TruGraph by @sfc-gh-dhuang in #2134

New Contributors

• @sfc-gh-ajia made their first contribution in #2108

Full Changelog: trulens-1.5.3...trulens-2.1.0