Support embedding model environmental cost

[252afh]

Description

I use ecologits for measuring chat_completion cost through LiteLLM, but I can't see a way to get costs for embedding via ecologits.

This is our current implementation in-progress i-dot-ai/i-dot-ai-utilities#8.

I want to use it to aggregate all of our carbon cost for all LLM calls. Currently I'm only able to get chat_completions, which feels like only half the pie.

I believe a lot of other people that use AI will need this for use in RAG systems especially, so it's worth embedding into ecologits for a single point-of-contact.

[adrienbanse]

Hey @252afh :)

Although I understand the frustration, for now, our methodology is limited to text generation.

As this feature would require developing a new methodology and implementing it, I don't think that it is a priority right now, but it would clearly be a good middle/long-term goal!

[252afh]

Hey @adrienbanse !

Thanks for the response, I'd be interested in staying updated on any progress you make on this but agree that it's not a small feat!

Do you know of any other packages that serve the same purpose as ecologits but for embedding costs? I'm looking to create a paper-trail of accountability for our carbon usage so keen to get the entire picture captured :)

Ell

[adrienbanse]

I'm not aware of any such software, sorry..

One comment though: I expect these costs to be much smaller than the text generation, but of course it depends on how much you request them.

We'll for sure let you know if such a feature is considered in Ecologits.