Releases: truera/trulens
Releases · truera/trulens
TruLens 1.4.6
What's Changed
- bump otel version by @sfc-gh-chu in #1871
- Sync conda deps with pyproject deps by @sfc-gh-chu in #1872
- Enable logging in MySQL by @calabozo in #1870
- Put costs in current span. by @sfc-gh-dkurokawa in #1873
- SDK: Fix run metadata field masks logic and expose description and label as optional and update-able fields by @sfc-gh-dhuang in #1868
New Contributors
Full Changelog: trulens-1.4.5...trulens-1.4.6
TruLens 1.4.5
What's Changed
- Fix double quotes issue in run operation system function / method by @sfc-gh-dhuang in #1853
- Get E2E tests running (yes, not even working just running). by @sfc-gh-dkurokawa in #1850
- Create load test. by @sfc-gh-dkurokawa in #1854
- Require newer versions of OTEL packages. by @sfc-gh-dkurokawa in #1855
- Update Snowflake OTEL E2E tests to use Snowflake runs. by @sfc-gh-dkurokawa in #1849
- SDK - app/ snowflake object name case resolution and fixing span attributes validation in dataset_spec by @sfc-gh-dhuang in #1856
- Make CUJ work with local LLM by @sfc-gh-pdharmana in #1858
- Remove mention of cortex guard tokens. by @sfc-gh-dkurokawa in #1860
- Remove requirement for ALTER SESSION as we don't need it anymore. by @sfc-gh-dkurokawa in #1865
- Call feedbacks with feedback.call by @sfc-gh-chu in #1864
- Update latest cortex cost table by @sfc-gh-dhuang in #1863
- Run compute metric returning statuses instead of async job by @sfc-gh-dhuang in #1866
Full Changelog: trulens-1.4.4...trulens-1.4.5
TruLens 1.4.4
What's Changed
- Add hotpotqa data to pupr notebook by @sfc-gh-chu in #1845
- Have all of the
TestOtelTruBasic
/TestOtelTruChain
/TestOtelTruCustom
/TestOtelTruLlama
be subclasses of a superclass that has shared tests for all of them. by @sfc-gh-dkurokawa in #1844 - Handle Japanese (and other non-ascii languages) better. by @sfc-gh-dkurokawa in #1848
- Check before trying to remove quotes if they enclose db and schema. by @sfc-gh-dkurokawa in #1847
- [SNOW-1943864] SDK Run status tracking for invocations, computations, and metrics by @sfc-gh-dhuang in #1840
- SDK: Map Span attributes to dataset_spec for Run + misc fixes by @sfc-gh-dhuang in #1851
Full Changelog: trulens-1.4.3...trulens-1.4.4
trulens-1.4.3
What's Changed
Bug Fixes
- Filipg/fix hotspots for release by @sfc-gh-fgralinski in #1821
- LiteLLM fixes by @sfc-gh-jreini in #1829
- Don't track feedback functions. by @sfc-gh-dkurokawa in #1833
- Use context variables instead of thread_local since the latter doesn't actually inherit values from the parent threads. by @sfc-gh-dkurokawa in #1838
- remove redundant header in streamlit pills by @sfc-gh-jreini in #1839
Docs
- change link from slack to snowflake discourse by @sfc-gh-jreini in #1835
- update the README with community link by @sfc-gh-jreini in #1841
Otel and other updates
- Fix issue where for
llama-index
apps, we don't handle the app specific record root correctly. by @sfc-gh-dkurokawa in #1810 - Redefine span attribute dictionary so that the values of the dictionary map to kwargs of the function decorated or the return. by @sfc-gh-dkurokawa in #1826
- Replace
attributes
withfull_scoped_attributes
. by @sfc-gh-dkurokawa in #1827 - Rename
main_input
/main_output
/main_error
toinput
/output
/error
. by @sfc-gh-dkurokawa in #1828 - SDK bugbash action items by @sfc-gh-dhuang in #1830
- SDK - run.describe() to return Run object directly #1831 by @sfc-gh-dhuang in #1832
- Clean up tests a bit and add a test to check if span attributes fail, then everything fails. by @sfc-gh-dkurokawa in #1837
- Handle the possible creation of
TruSession
whenconnector
argument is given to the tru app. by @sfc-gh-dkurokawa in #1842 - SDK run.start() should take ground_truth_output from dataset_spec by @sfc-gh-dhuang in #1843
Full Changelog: trulens-1.4.2...trulens-1.4.3
TruLens 1.4.2
What's Changed
- [SNOW-1901834] SDK: Run APIs: CREATE, GET, LIST, and DELETE and run.start() by @sfc-gh-dhuang in #1784
Full Changelog: trulens-1.4.1...trulens-1.4.2
Trulens v1.4.1
What's Changed
- Bugfix: Avoid ingestion of incorrectly generated spans. by @sfc-gh-apgupta in #1809
- Clean up semantic conventions. by @sfc-gh-dkurokawa in #1811
- Don't output a lot of span attributes if they're
None
. by @sfc-gh-dkurokawa in #1812 - Set default host for snowflake connector if not provided by @sfc-gh-dhuang in #1814
- Test conda build in PR pipeline. by @sfc-gh-dkurokawa in #1813
- Ensure there's only one record root span and also discern the main method if none given for
TruApp
/TruCustomApp
. by @sfc-gh-dkurokawa in #1818 - OTEL -> Otel to follow general naming conventions and keep things consistent. by @sfc-gh-dkurokawa in #1819
New Contributors
- @sfc-gh-apgupta made their first contribution in #1809
Full Changelog: trulens-1.4.0...trulens-1.4.1
TruLens v1.4.0
What's Changed
- Create very general function to compute feedbacks. by @sfc-gh-dkurokawa in #1794
- even quieter logging by @sfc-gh-jreini in #1795
- Create distributed OTEL test. by @sfc-gh-dkurokawa in #1789
- Trulens hotspots by @sfc-gh-fgralinski in #1757
- Clean up tests a bit and remove
ai.observability.domain
as a span attribute. by @sfc-gh-dkurokawa in #1796 - Update semantic convention for EVAL and EVAL_ROOT and update test to use this. by @sfc-gh-dkurokawa in #1797
- SDK app wrapper (EXTERNAL_AGENT ) CRUD by @sfc-gh-dhuang in #1772
- Create Snowflake E2E version of the feedback computation test. by @sfc-gh-dkurokawa in #1798
- Allow for
ground_truth_output
to be specified to recorder and also clean up semantic conventions by @sfc-gh-dkurokawa in #1799 - add llamaindex/langchain conda build recipes by @sfc-gh-chu in #1767
- Create OTEL span processor that can add trulens span attributes into spans not originated by trulens. by @sfc-gh-dkurokawa in #1800
- Make main_method required only for TruChain and TruLlama by @sfc-gh-dhuang in #1802
- Set up new context managers to record. by @sfc-gh-dkurokawa in #1805
New Contributors
- @sfc-gh-fgralinski made their first contribution in #1757
Full Changelog: trulens-1.3.5...trulens-1.4.0
TruLens 1.3.5
What's Changed
- Fix
snowflake-sqlalchemy
autocommit issue. by @sfc-gh-dkurokawa in #1792
Full Changelog: trulens-1.3.4...trulens-1.3.5
TruLens 1.3.4
What's Changed
- Validation for sis dashboard names by @sfc-gh-chu in #1750
- Handle sync and async generators with the OTEL
@instrument
decorator. by @sfc-gh-dkurokawa in #1748 - Require version of snowflake ml python >= 1.7.2 by @sfc-gh-dhuang in #1752
- Write returns for "UNKNOWN" spans. by @sfc-gh-dkurokawa in #1753
- Set up basic E2E test. by @sfc-gh-dkurokawa in #1754
- Handle sync functions that aren't generators but return a generator. by @sfc-gh-dkurokawa in #1755
- Track OpenAI and Cortex costs in a similar manner to the non-OTEL world. by @sfc-gh-dkurokawa in #1756
- Remove some old OTEL logic. by @sfc-gh-dkurokawa in #1759
- Have llama-index emit context-retrieval spans. by @sfc-gh-dkurokawa in #1761
- Change base scope from "ai_observability" to "ai.observability". Also… by @sfc-gh-dkurokawa in #1762
- Test guardrails with OTEL setup. by @sfc-gh-dkurokawa in #1764
- Make OTEL flow for users more natural by only having to set an environment variable and importing from non-experimental code. by @sfc-gh-dkurokawa in #1766
- Add in Snowflake OTEL e2e tests for tru_chain and tru_llama. by @sfc-gh-dkurokawa in #1768
- clean up old benchmark notebooks by @sfc-gh-jreini in #1760
- Migrate TruCustomApp to TruApp (backward compatible) by @sfc-gh-dhuang in #1770
- Set kwargs, return, and exception on all spans in the
ai.observability.call
span attribute scope. by @sfc-gh-dkurokawa in #1777 - Patch groundedness configs by @sfc-gh-jreini in #1778
- fix broken link on homepage by @sfc-gh-jreini in #1779
- Disable python <3.9 by @sfc-gh-chu in #1780
- Add in some minor test improvements. by @sfc-gh-dkurokawa in #1769
- make main_method required for app base class for OTel by @sfc-gh-dhuang in #1771
- Quieter Instrumentation by @sfc-gh-jreini in #1783
- Create OTEL notebook tests. by @sfc-gh-dkurokawa in #1782
- Run an E2E notebook on Snowflake notebooks (via staging the trulens packages). by @sfc-gh-dkurokawa in #1785
- Cost track
litellm.completion
. by @sfc-gh-dkurokawa in #1781 - Use new SPROC and UDTF, and verify more for OTEL Snowflake exporter. by @sfc-gh-dkurokawa in #1786
- Use
pytest
for grouping required/optional tests. by @sfc-gh-dkurokawa in #1788 - Remove
pytest.ini
as it's interfering with thepyproject.toml
. by @sfc-gh-dkurokawa in #1790
Full Changelog: trulens-1.3.3...trulens-1.3.4
TruLens 1.3.3
What's Changed
- Enable exporting spans to snowflake stage if a
TruLensSnowflakeSpanExporter
is provided by @sfc-gh-gtokernliang in #1708 - Allow
TruChain
/TruLlama
/TruRails
to use OTEL spans. by @sfc-gh-dkurokawa in #1727 - move
poetry-core
restrictions to az pipeline by @sfc-gh-chu in #1732 - Updated context relevance chain-of-thought prompting by @sfc-gh-dhuang in #1744
- Add Llama-Index support for OTEL by @sfc-gh-dkurokawa in #1743
- Write args as kwargs to unknown type OTEL spans by @sfc-gh-chu in #1745
- Add async support for OTEL by @sfc-gh-dkurokawa in #1746
- Add multithreading test for OTEL by @sfc-gh-dkurokawa in #1747
Bug Fixes
- Fix type hint for fewshot prompt construction by @sfc-gh-dhuang in #1725
- Add protobuf-related utilities by @sfc-gh-gtokernliang in #1720
- fix typo in snowflake feedbacks notebook by @sfc-gh-jreini in #1731
- groundedness: update user prompt to be consistent with system prompt by @sfc-gh-jreini in #1730
run_dashboard
to print Local URL by @sfc-gh-jreini in #1740
Full Changelog: trulens-1.3.2...trulens-1.3.3