Skip to content

Improve tool call by re-trying tool call with the model response content #1052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

briandevvn
Copy link

During tool calling, the model sometimes does not call any tools, instead, it returns the string content with the JSON arguments or text that explains what tools are using and the arguments.

This PR takes care of this case by retrying calling tools with the model response content because the response content contains the tool call name and the argument. We can use the model to extract the value from that.

Example:

Output message of the LLM: ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
ModelResponse(id='chatcmpl-16ff14bb-1aff-427b-8ed8-0ddbc4a65d64', created=1742489273, model='ollama_chat/llama3.1:latest', object='chat.completion', system_fingerprint=None,                       
choices=[Choices(finish_reason='stop', index=0, message=Message(content='[ACTION]:\n```\nFirst action: Visit apple.com\n```python\nvisit_page(url="https://apple.com/")\n```',          
role='assistant', tool_calls=None, function_call=None, provider_specific_fields=None))], usage=Usage(completion_tokens=30, prompt_tokens=2080, total_tokens=2110, completion_tokens_details=None,   
prompt_tokens_details=None))                                                                                                                                                                        
Model did not call any tools. Retrying with the model response content... ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[ACTION]:                                                                                                                                                                                                                                                                                                                                                                                      
First action: Visit apple.com                                                                                                                                                                 
python                                                                                                                                                                                           
visit_page(url="https://apple.com/")                                                                                                                                                          
                                                                                                                                                                                                 
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'visit_page' with arguments: {'url': 'https://apple.com/'}                                                                                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                                

@@ -1019,6 +1019,44 @@ def step(self, memory_step: ActionStep) -> Union[None, Any]:
level=LogLevel.DEBUG,
)

# Re-try if the model did not call any tools
if (model_message.content != '') and (model_message.tool_calls is None or len(model_message.tool_calls) == 0):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can write:

if (model_message.content.strip() != '') and not model_message.tool_calls:
    ...

The not model_message.tool_calls covers both None and empty list cases.

You should strip content to cover the case that content has some data but only whitespaces.

stop_sequences=["Observation:", "Calling tools:"],
)
memory_step.model_output_message = model_message
except Exception as e:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catch a concrete exception class, not general Exception class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants