Outlet Langfuse Logging with chat_id Retention #430
Conversation
|
Thank you, this is working for me. |
|
|
||
| trace = self.langfuse.trace( | ||
| name=f"filter:{__name__}", | ||
| input=body, |
There was a problem hiding this comment.
Should last message be excluded from the input? If it is executed on the outlet, the body will contain every previous message plus the last generated one, which should not be part of the input. Same logic applies in the generation.
There was a problem hiding this comment.
Thank you for the important feedback!
For now, I’ve addressed the issue by removing only the last message. I believe this approach is generally sufficient, but as you pointed out, it may not cover cases like retries or supplemental outputs where multiple assistant messages appear consecutively.
While it is possible to go back further, I chose this simpler approach to keep the example code straightforward. Depending on the application, the handling of messages may vary, but I believe this is sufficient as an example. If you have other approaches in mind, I’d be glad to discuss them as well.
There was a problem hiding this comment.
I think it fits the general behaviour, thanks for your amazing work <3
|
|
||
| if ( | ||
| body["chat_id"] not in self.chat_generations | ||
| or body["chat_id"] not in self.chat_traces |
There was a problem hiding this comment.
Is this cache needed? If the generation and trace objects are stored in the dictionary, retrieved and removed in the same method, can it just reuse the objects without the dictionaries?
There was a problem hiding this comment.
I realized that I was using the cache out of habit from the original code, but as you pointed out, using variables is sufficient for this implementation. I’ve simplified the code accordingly. Thanks for helping me improve it.
… from input to maintain context integrity
|
@JTHesse @ADD-Carlos-Zamora I’m glad to hear that it's working for you. Regarding the usage information not showing up, it seems to be a separate issue unrelated to this pipeline. |
According to Issue #415, in the latest version, the pipe function is executed before the inlet function, resulting in the following problems:
Problems:
The inlet is expected to extract the chat_id from the request body and add or modify it in the body. However, since the pipe executes first, this process is not completed, leaving the chat_id absent.
The pipe function attempts to extract the chat_id from the body for processing, but without the correct chat_id, inconsistencies occur in log retrieval and various processes. Additionally, there is an issue where the inlet is not being called properly in the first place.
Proposal: To address the problem where the pipe function initiates before the chat_id extraction process in the inlet, we propose changing the log generation method to the outlet side. With this change, the chat_id will be correctly preserved on the outlet side, ensuring that Langfuse logs are reliably retrieved. Although future adjustments to the execution order and configurations should be considered, this approach of retrieving logs on the outlet side is proposed as a way to bypass the current issue.
Change Details:
Changed the implementation to generate Langfuse logs on the outlet side, removing dependency on the chat_id extraction process in the inlet.
Added processing to reliably obtain the chat_id from request information during log generation on the outlet and attach it to the log.
Verification:
Verified that the changes do not affect existing pipeline processing and that logs are retrieved correctly.