Skip to content

Conversation

@ainformatico
Copy link

Motivation

This fixes an issue where the order of agent names was not preserved due to using a set. This names are being used when preparing the tool node, and are present on the final prompt to the LLM. On every run, the order was different causing issues when using LANGSMITH_TEST_CACHE=path/to/cassets.

image As seen on the screenshot above, each run was giving a different order.

This fix has reduced significantly the amount of real HTTP calls executed.

Questions

  • I could not find an easy way to add a test to this as the final prompt is quite buried in abstractions. Would love to hear from you an approach for this. Claude and Qwen also tried but failed.
  • We still have some real HTTP calls due to the tool.id being different on every run. Any thoughts on this?

This fixes an issue where the order of agent names was not preserved due to using a set.
This names are being used when preparing the tool node, and are present on the final prompt to the LLM.
On every run, the order was different causing issues when using `LANGSMITH_TEST_CACHE=path/to/cassets`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant