Open
Description
Is your feature request related to a problem? Please describe.
I would like to know what my pipeline costs and have data to effectively improve it.
Describe the solution you'd like
After a pipeline completes, I should have access to numbers of nodes, errors, usage per model/llm, perhaps more. Would be cool if part of this data is also available via tracing.
Some musings I had on this:
- Separate aggregation thread or atomics?
- Would be nice if we can get the data while its running (apps can then display progress)
- There might be a way to estimate the total amount of nodes at any stage (depending on the type of stream)
- Also nice if the same data can be used to enhance tracing, preferably matching the beta otel llm spec