Skip to content

Thread Local and Async Local history for BaseLM#9261

Open
TyTodd wants to merge 3 commits intostanfordnlp:mainfrom
TyTodd:tyrin/safe-history
Open

Thread Local and Async Local history for BaseLM#9261
TyTodd wants to merge 3 commits intostanfordnlp:mainfrom
TyTodd:tyrin/safe-history

Conversation

@TyTodd
Copy link

@TyTodd TyTodd commented Feb 6, 2026

I've ran into this use case a couple times now where I need to get the history of a Predict but feel uneasy about using LM.history because the same LM is being used in multiple coroutines. The devx for getting a thread safe view of history is a little verbose, you either have to use dspy.context with a copy of the global LM. Or run a separate copy of your module if it has its self.lm. In some (rare) cases even this is not enough. I recently ran into a scenario where I needed to use the same module across multiple co-routines. The module had its own self.lm so using dspy.context wasn't a viable solution since it injects the LM into the local dspy.settings.

This short PR adds thread/async-local history to LM via a ContextVar called local_history. Allowing users to view the history from the local thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments