Regarding tutorial '25_Customizing_Agent' (3)

[dkbs12]

Hello,
This is the third inquiry about the tutorial '25_Customizing_Agent'.
I think there are many things that are suitable for practical application and that's why I can't stop making questions.
I have many questions at this time as below;

1. It always need to spend a long time to get answer from Agent chatting.
   It takes about from 10 to 20 seconds to get feedback from Agent and it's much longer than the time from Generative QA Pipeline.
   (Generative QA Pipeline needs average 2 or 3 seconds to produce an answer.)
   Is there any solution to reduce the feedback time from Agent?

2. I can't get the score, context and the name of file from the answer made by Generative QA Pipeline as below.
   How can I get those information from Generative QA Pipeline when I run print_answer function?

   < Question >
   response = generative_pipeline.run("What does Rhodes Statue look like?")
   print_answers(response, details="minimum")

   < Answer >
   'Query: What does Rhodes Statue look like?'
   'Answers:'
   [ { 'answer': 'The documents do not provide a description or image of what '
   'the Rhodes Statue looked like. Answering the question is '
   'not possible given the available information.'}]

   I can't get the score, context, the name if file and document from from the answer made by Agent, either.
   Thew are only things like observation and transcript that I can find in the response.
   But, I think those information as score, context, the name of file and document are more useful for the users.
   Is there any solution to display them in the response from the Agent?

3. When I try a question in local language during Agent chatting, the correct answer comes out in English.
   But, When I request Agent to answer in local language, it produce the answer 'KeyError: 'translation'.
   In addition, I ask Agent to translate the answer in English to local language and then Agent make an error as "Input: Translate the previous answer into Korean"
   How can I use local language in question and answer when using the Agent chatting?

Thank you for your help in advance.

[bogdankostic]

It always need to spend a long time to get answer from Agent chatting.
It takes about from 10 to 20 seconds to get feedback from Agent and it's much longer than the time from Generative QA Pipeline.
(Generative QA Pipeline needs average 2 or 3 seconds to produce an answer.)
Is there any solution to reduce the feedback time from Agent?

The agent uses a large language model to imitate a thought process. During this process, it makes multiple requests to the language model. One of its steps is to execute the generative QA pipeline. As a result, using the agent on your query will always take longer than using the generative QA pipeline tool directly on your query.

I can't get the score, context and the name of file from the answer made by Generative QA Pipeline as below.
How can I get those information from Generative QA Pipeline when I run print_answer function?

You can set details="all" to see the IDs of the documents that were used to generate the answer. You can use theses IDs to fetch the documents from the document store and get the metadata you're interested in. Unfortunately, we don't provide a score answers coming from generative models.

I can't get the score, context, the name if file and document from from the answer made by Agent, either. Is there any solution to display them in the response from the Agent?

We can't provide a score for answers generated by an Agent either. Also, we don't provide document IDs for Agents.

When I try a question in local language during Agent chatting, the correct answer comes out in English.
But, When I request Agent to answer in local language, it produce the answer 'KeyError: 'translation'.
In addition, I ask Agent to translate the answer in English to local language and then Agent make an error as "Input: Translate the previous answer into Korean"
How can I use local language in question and answer when using the Agent chatting?

Can you give more details about this? How do you instruct the Agent to output the answer in a different language? Which models are you using? A code example would be very helpful here.

[dkbs12]

HI, bogdankostic.
Thank you for your kind explanation.
As for the first inquiry, it's a downside of Agent to need a little bit long time to get an answer, although it has a powerful function as a chat bot.
My purpose is to avoid the worst case to exceed 20 seconds to get an answer from Agent and those cases is not so many because it happens when it finds the answer to the end even though there is no answer in the tools.
I know the number of iterations of using tools is limited to eight by default.
Can I reduce the time to get answer from Agent by set the max_steps parameter to only four?

As for the last issue, the local language questioning and answering, there is nothing special to let the Agent answer in local language or understand local language.
I guess Agent which is made of OpenAI could understand the local language just as ChatGPT does, and it worked.
Every thing was as same as Tutorial #25, model for generative QA pipeline was 'text-davinci-003' and model for Agent was gpt-3.5-turbo.
And I made questions to Agent just as below;

Q1 : conversational_agent.run("로도스 동상은 어떻게 생겼나요?") => It means "What did Rhodes Statue look like?" in Korean.
Final Answer: The Rhodes Statue, or Colossus of Rhodes, was a 33-meter tall statue of the Greek sun-god Helios with curly hair and spikes of bronze or silver flame radiating from its head.'

Q2 : conversational_agent.run("좀 더 자세히 설명해 주세요.") => It means "Tell me more about it." in Korean.
Final Answer: The Rhodes Statue, or Colossus of Rhodes, was a 32-meter tall statue with iron tie bars and brass plates forming its skin. It stood on a 15-meter-high white marble pedestal near the entrance of the Rhodes harbor. The interior of the statue was filled with stone blocks.'

Q3: conversational_agent.run("방금 한 답변을 한국어로 번역해 주세요.") => It means "Please translate the answer you just gave into Korean." in Korean.
Answer : There were eight times of this phrase, 'Tool: Google Translate\nTool Input: "Translate the previous answer into Korean", inthe trascript part.

Q4 : conversational_agent.run("What did Rhodes Statue look like? Please answer in Korean.")
Final Answer: The Rhodes Statue, or the Colossus of Rhodes, was a 33-meter tall statue of the Greek sun-god Helios.'

Above is all that I know about local language questioning and answering.
If you need more details or information, please don't hesitate to ask me.
Thanks.

[bogdankostic]

Can I reduce the time to get answer from Agent by set the max_steps parameter to only four?

Yes, setting max_steps to 4 will lower the number of maximum tool executions per query, so this should reduce the execution time for queries that otherwise use more than 4 tool executions.

Regarding answering in Korean instead of English: I think for this you need to experiment a bit with the prompts. One thing you could try is to add an instruction to the agent prompt defined here%20Define%20the%20Prompt) that the Agent should answer the question in Korean (or in the language the question was raised by the user.)

[dkbs12]

HI, bogdankostic.
Thank you for your quick reply.
I agree with you to need to experiment with the prompt which was used in tutorial #25.
But, I'm not sure it will work or not, because know prompt engineering needs a sophiscated skill as you know.
If it succeeds, I'll let you know the result.

By the way, what about the Q3 example?
Do I need to make a specified tool for translation? (maybe something to be instructed to use Google tralslator or something?)
Is there any example for translation tool?

Thank you for your help.