Expand observability events to capture LLM input/output per turn #174
astrobot-houston
started this conversation in
Feature Request
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Adds an
turn_startobservability event emitting provider, model, inputs (system prompt, messages, tools) and reasoning level. Additionally expands theturnevent to include used provider, LLM outputHaving such an event is useful for AI observability, helping understand exactly what the LLM is thinking between turns.
Implementation wise, the typings for
toTurnMessageandtoTurnContentwhich are new mapping functions, turning Pi output into Flue event data, are a bit ugly.The other thing I am not 100% about is the new private
activeTurnIdfield onSession. It relies heavily on the fact that there con only be one active turn for a given session. I don't know if that might become a problem in the future.Original implementation from #173 by @lforst
Beta Was this translation helpful? Give feedback.
All reactions