-
Notifications
You must be signed in to change notification settings - Fork 228
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Bug Description
I encountered a UnicodeEncodeError while defining the feedback function
To Reproduce
from trulens.core import TruSession
from trulens.core import Feedback
from trulens.apps.llamaindex import TruLlama
from trulens.providers.langchain import Langchain
from trulens.feedback.v2.feedback import Groundedness
from langchain_community.llms import SparkLLM
tru = TruSession()
tru.reset_database()
langc = Langchain(chain=spark_llm)
qa_relevance = (
Feedback(langc.relevance_with_cot_reasons, name="Answer Relevance")
.on_input_output()
)
qs_relevance = (
Feedback(langc.relevance_with_cot_reasons, name="Context Relevance")
.on_input()
.on(TruLlama.select_source_nodes().node.text)
.aggregate(np.mean)
)
groundedness = (
Feedback(langc.groundedness_measure_with_cot_reasons, name="Groundedness")
.on(TruLlama.select_source_nodes().node.text)
.on_output()
)
feedbacks = [qa_relevance, qs_relevance, groundedness]
tru_recorder = TruLlama(
query_engine,
app_id=app_id,
feedbacks=feedbacks
)
Expected behavior
It should not report an error, but instead obtain a result similar to the following:
✅ In Groundedness, input source will be set to __record__.app.first.steps__.context.first.invoke.rets[:].page_content.collect() .
✅ In Groundedness, input statement will be set to __record__.main_output or `Select.RecordOutput` .
✅ In Answer Relevance, input prompt will be set to __record__.main_input or `Select.RecordInput` .
✅ In Answer Relevance, input response will be set to __record__.main_output or `Select.RecordOutput` .
✅ In Context Relevance, input question will be set to __record__.main_input or `Select.RecordInput` .
✅ In Context Relevance, input context will be set to __record__.app.first.steps__.context.first.invoke.rets[:].page_content .
Relevant Logs/Tracebacks
The error message is as follows:
UnicodeEncodeError Traceback (most recent call last)
Cell In[24], line 4
1
2 qa_relevance = (
3 Feedback(langc.relevance_with_cot_reasons, name="Answer Relevance")
----> 4 .on_input_output()
5 )
7 qs_relevance = (
8 Feedback(langc.relevance_with_cot_reasons, name="Context Relevance")
9 .on_input()
10 .on(TruLlama.select_source_nodes().node.text)
11 .aggregate(np.mean)
12 )
15 groundedness = (
16 Feedback(langc.groundedness_measure_with_cot_reasons, name="Groundedness")
17 .on(TruLlama.select_source_nodes().node.text)
18 .on_output()
19 )
File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:296, in Feedback.on_input_output(self)
289 def on_input_output(self) -> Feedback:
290 """
291 Specifies that the feedback implementation arguments are to be the main
292 app input and output in that order.
293
294 Returns a new Feedback object with the specification.
295 """
--> 296 return self.on_input().on_output()
File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:591, in Feedback.on_prompt(self, arg)
589 if arg is None:
590 arg = self._next_unselected_arg_name()
--> 591 self._print_guessed_selector(arg, select_schema.Select.RecordInput)
593 new_selectors[arg] = select_schema.Select.RecordInput
595 ret = self.model_copy()
File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:323, in Feedback._print_guessed_selector(self, par_name, par_path)
320 else:
321 alias_info = ""
--> 323 print(
324 f"{text_utils.UNICODE_CHECK} In {self.supplied_name if self.supplied_name is not None else self.name}, "
325 f"input {par_name} will be set to {par_path}{alias_info} ."
326 )
File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Scripts\pywin32_postinstall.py:27, in Tee.write(self, what)
25 except OSError:
26 pass
---> 27 tee_f.write(what)
UnicodeEncodeError: 'gbk' codec can't encode character '\u2705' in position 0: illegal multibyte sequence
Environment:
- OS: Windows
- Python Version: 3.12
- TruLens version:1.3.3
dosubot
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working