Skip to content

benchmarks

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

benchmarks

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
collect_output_lengths
collect_output_lengths
 
 
kernels
kernels
 
 
MY_README.md
MY_README.md
 
 
README.md
README.md
 
 
add_a_new_model.md
add_a_new_model.md
 
 
benchmark_latency.py
benchmark_latency.py
 
 
benchmark_serving.py
benchmark_serving.py
 
 
benchmark_throughput.py
benchmark_throughput.py
 
 
check_res_accuracy.py
check_res_accuracy.py
 
 
comp_model_size.py
comp_model_size.py
 
 
construct_cost_model.py
construct_cost_model.py
 
 
draw_inp_len_distribution.py
draw_inp_len_distribution.py
 
 
fake_scheduling.py
fake_scheduling.py
 
 
get_my_cost_table_directly.py
get_my_cost_table_directly.py
 
 
get_my_cost_table_directly2.py
get_my_cost_table_directly2.py
 
 
launch_tgi_server.sh
launch_tgi_server.sh
 
 
model_coeff_database.py
model_coeff_database.py
 
 
model_initcost_database.py
model_initcost_database.py
 
 
model_size_database.py
model_size_database.py
 
 
my_llm_infer_worker_multiprocessing.py
my_llm_infer_worker_multiprocessing.py
 
 
my_per_iter_latency_estimator.py
my_per_iter_latency_estimator.py
 
 
no_robot_test.parquet
no_robot_test.parquet
 
 
no_robot_train.parquet
no_robot_train.parquet
 
 
output_length_sampler.py
output_length_sampler.py
 
 
router_bench_multiple_choice_question_dataset.json
router_bench_multiple_choice_question_dataset.json
 
 
router_bench_multiple_choice_question_dataset_with_responses.json
router_bench_multiple_choice_question_dataset_with_responses.json
 
 
router_bench_not_multiple_choice_question_dataset.json
router_bench_not_multiple_choice_question_dataset.json
 
 
router_bench_not_multiple_choice_question_dataset_with_responses.json
router_bench_not_multiple_choice_question_dataset_with_responses.json
 
 
run_end2end_exp.sh
run_end2end_exp.sh
 
 
schedule_multi_model.py
schedule_multi_model.py
 
 
search_exec_plans.py
search_exec_plans.py
 
 
train-00000-of-00001-b334c773bce22cb2.parquet
train-00000-of-00001-b334c773bce22cb2.parquet
 
 

README.md

Downloading the dataset

You can download the dataset by running (the datasets for router and chain summary are included in this repo):

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
wget https://huggingface.co/datasets/liyucheng/arxiv-march-2023/resolve/main/data/train-00000-of-00001-b334c773bce22cb2.parquet

wget https://huggingface.co/datasets/llm-blender/mix-instruct/resolve/main/train_data_prepared.jsonl

wget https://huggingface.co/datasets/kmfoda/booksum/resolve/main/train.csv