Skip to content

Conversation

@changliu2
Copy link

@changliu2 changliu2 commented Feb 25, 2025

To get a diverse set of agent scenario data, run batch-agent.py for batch inference to cover these scenarios:

A general Q&A agent with no tools
A web search agent that has the Bing Search tool configured and provides information about the weather
A simple RAG agent with file search tool configured and some documents added
A query that requires a custom function call with stock data
An agent with access to Open Weather API

Notes

  1. the PoC converter will also transform data into messages which includes enough information for evaluation).
  2. The converter showcase what data is needed for evaluation.

Caveats:

  1. hard-coded weather call to avoid auth errors (perhaps I don't have the right api key). So questions about the weather call will always return the same response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant