[FEATURE] Create a custom ArgillaNode to monitor Nodes/Components #3
Description
Is your feature request related to a problem? Please describe.
With the current structure of Deepset Haystack, there is the possibity to use callback only with Agents. This prevents obtaining inner processes of the workflow. This PR #2 adds an ArgillaCallbackHandler to log LLM results and some metadata into Argilla with no other info.
Describe the solution you'd like
A custom ArgillaNode or a wrapper can be introduced to account for other processes as proposed here.
Describe alternatives you've considered
It is possible that this solution might not be ideal for some cases. For such a case, if a general CallbackManager is introduced by Deepset, then we can make use of it to further improve the integration between Argilla and Haystack.
Additional context
One thing to keep in mind is that Deepset is planning to introduce Haystack 2.0, which is currently in beta version. The changes here might be relevant for our work, like the change from Nodes to Components. More info on this page.