Description
Description:
We will need to create the standardize profiles what sizing and create all of the relevant files/folders and land them in easily understandable, organized folder structures w/in GuideLLM to offer users the ability to benchmark different scenarios on GuideLLM. These profiles should cover the very specific cases that a performance team may want to run to understand model performance under various dataset loads. This set of data should be comprehensive and offer great flexibility in benchmarking models under a wide range of dataset breakdowns.
User Story
Additional Docs
Acceptance Criteria
Create standard dataset profiles per: https://docs.google.com/document/d/1Ql6RI3_LbhQxqFCIu_n2A2CewK6OVVFZFYgu4KEjjtw/edit?usp=sharing
Land these profiles in a 'dataset-profiles' folder under the GuideLLM upstream
folder formatting:
- README - describes the profiles below and the use cases they cover
- SI/MO
- SI/LO
- MI/MO
- LI/SO
- LI/MO
- LI/LO
- MI/XO
- XI/MO
Metadata
Metadata
Assignees
Labels
Type
Projects
Status