Help with using Feedback().on().

[tfs-hendricks]

https://www.trulens.org/reference/trulens/providers/litellm/provider/#trulens.providers.litellm.provider.LiteLLM.trajectory_logical_consistency_with_cot_reasons

When trying to run

f_logical_consistency = (
    Feedback(provider.trajectory_logical_consistency_with_cot_reasons)
    .on({
        "trace": Selector(trace_level=True),
    })

I keep getting the error
ValueError: Expected a Lens but got Selector(trace_level=True, function_name=None, span_name=None, span_type=None, span_attributes_processor=None, span_attribute=None, function_attribute=None, ignore_none_values=False, collect_list=True, match_only_if_no_ancestor_matched=False) of type Selector.

How do I set this correctly so that it will take the evaluate the trajectory of the agent?

[dosubot[bot]]

The error happens because Feedback().on() expects a Lens, not a Selector. To fix this, wrap your Selector in a Lens using Lens.from_selector. Here’s how you can do it:

from trulens_eval.schema.lenses import Lens
from trulens_eval.schema.selector import Selector

f_logical_consistency = (
    Feedback(provider.trajectory_logical_consistency_with_cot_reasons)
    .on({
        "trace": Lens.from_selector(Selector(trace_level=True)),
    })
)

This will let Feedback extract the agent trajectory correctly for evaluation. Passing a raw Selector only works in OTEL mode, which is not the default and not recommended for most users. For standard usage, always convert your Selector to a Lens before passing it to .on() source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[tfs-hendricks]

@dosu
Those modules do not exist.

[dosubot[bot]]

The modules trulens_eval.schema.lenses and trulens_eval.schema.selector were removed or renamed in recent major updates. In the current codebase, you should import Lens from trulens.core.utils.serial and Selector from trulens.core.feedback.selector. However, for Feedback().on() in standard (non-OTEL) mode, you must use a Lens, not a Selector—there is no Lens.from_selector method.

To target the agent trajectory, construct a Lens pointing to the relevant field. For example:

from trulens.core.utils.serial import Lens

f_logical_consistency = (
    Feedback(provider.trajectory_logical_consistency_with_cot_reasons)
    .on(trace=Lens.of_string("trace"))
)

If your data structure is different, adjust the string in Lens.of_string accordingly. Passing a Selector directly only works in OTEL mode, which is not the default. This change is due to recent breaking changes in TruLens 2.x+ that restructured the modules and APIs.
Reference: Lens implementation and Feedback.on() usage

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}