Add evals with Inspect guide to Inference Providers#2036
Add evals with Inspect guide to Inference Providers#2036Vaibhavs10 merged 10 commits intohuggingface:mainfrom
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
davanstrien
left a comment
davanstrien
left a comment
There was a problem hiding this comment.
Very nice! Love the example! Made a few small comments and nit suggestion for language (feel free to ignore!)
Co-authored-by: Daniel van Strien <davanstrien@users.noreply.github.com>
Co-authored-by: Daniel van Strien <davanstrien@users.noreply.github.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
|
great blog post ! Don't know if it's in scope of the blog but lighteval is moving towards using inspect-ai as well. I'm adding a few utilities for comparing and evaluating hf-inference-providers. For example I added |
I had the same thought, not sure if in scope either. Maybe worth adding just a sentence with a link, but your call (of course) @dvsrepo |
@NathanHB, it would be awesome to add a reference/references to lighteval, even at each section (example) level. Would you be open to add some changes, or you prefer I add them? |
|
Added a small line at the end, feel free to rephrase if needed :) |
Great @NathanHB ! I rephrased it a bit and moved it to the second bullet point, let me know if the working seems right to you |
|
we're all set from my side, thanks a lot for the reviews @pcuenca @davanstrien @NathanHB ! |
|
Merged this - open PRs are detrimental to progress ^^ (we can iterate afterwards if needed) |
Thanks for merging @Vaibhavs10 ! |
6 participants
No description provided.