Unitxt 1.26.0 - Multi Threading
Main changes:
- Made Unitxt Thread-Safe so it can run in multi-threaded environments.
- Added an option to set sampling seed for demos (in context example). This is done by
demos_sampling_seed. It allows running the same dataset with different demo examples. - Improved printouts of instance scores with to_markdown() and summary in Unitxt. For example :
results = evaluate(predictions=predictions, data=dataset)
print(results.instance_scores.summary)
All changes:
- Add to_markdown() to InstanceScores to pretty print output by @yoavkatz in #1846
- Improved InstanceScores summary to be readible and in decent width by @yoavkatz in #1847
- Improve multi turn tool calling example by @elronbandel in #1848
- Add metrics documentation including range, directionality and references by @elronbandel in #1850
- Fix sacrebleu documentation by @elronbandel in #1851
- Add F1 score documentation to F1Fast metric class by @elronbandel in #1852
- Add more llmjudge benchmarks by @martinscooper in #1804
- Fix llama scout name and url on rits by @martinscooper in #1857
- Add demos_sampling_seed to recipe api by @elronbandel in #1858
- Add comprehensive multi threading support and tests by @elronbandel in #1853
- Update BlueBench to match the original implementation by @bnayahu in #1855
Full Changelog: 1.25.0...1.26.0