Skip to content

Text-dataset performance profile creation - decide what the profiles are.  #82

Open
@rgreenberg1

Description

@rgreenberg1

Description:
We will need to create the standardize profiles what sizing and create all of the relevant files/folders and land them in easily understandable, organized folder structures w/in GuideLLM to offer users the ability to benchmark different scenarios on GuideLLM. These profiles should cover the very specific cases that a performance team may want to run to understand model performance under various dataset loads. This set of data should be comprehensive and offer great flexibility in benchmarking models under a wide range of dataset breakdowns.

User Story

Additional Docs

Acceptance Criteria
Create standard dataset profiles per: https://docs.google.com/document/d/1Ql6RI3_LbhQxqFCIu_n2A2CewK6OVVFZFYgu4KEjjtw/edit?usp=sharing
Land these profiles in a 'dataset-profiles' folder under the GuideLLM upstream
folder formatting:

  • README - describes the profiles below and the use cases they cover
  • SI/MO
  • SI/LO
  • MI/MO
  • LI/SO
  • LI/MO
  • LI/LO
  • MI/XO
  • XI/MO

Metadata

Metadata

Assignees

Labels

datasetDataset workstream

Type

Projects

Status

Ready

Relationships

None yet

Development

No branches or pull requests

Issue actions