Skip to content

Commit fe22b2e

Browse files
committed
Minor fixes. Changes to table of contents and required resources moved up
Minor fixes. Changes to table of contents and required resources moved up Signed-off-by: Aryan <nandaaryan823@gmail.com> grammar corrected and table of contents changed Signed-off-by: Aryan <nandaaryan823@gmail.com>
1 parent 51b661c commit fe22b2e

1 file changed

Lines changed: 26 additions & 23 deletions

File tree

  • examples/cloud-edge-collaborative-inference-for-llm

examples/cloud-edge-collaborative-inference-for-llm/README.md

Lines changed: 26 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,11 @@
55
- [Why LLM Need Cloud-Edge Collaborative Inference?](#why-llm-need-cloud-edge-collaborative-inference)
66
- [Possible Collaborative Inference Strategy](#possible-collaborative-inference-strategy)
77
- [Details of Design](#details-of-design)
8-
- [Quick Start Guide for Benchmarking with Ianvs](#quick-start-guide-for-benchmarking-with-ianvs)
8+
- [Quick Start Guide](#quick-start-guide)
9+
- [Required Resources](#required-resources)
10+
- [Methods for benchmarking with ianvs](#methods-for-benchmarking-with-ianvs)
911
- [Docker-Based Setup](#docker-based-setup)
1012
- [Detailed Setup Guide](#detailed-setup-guide)
11-
- [Required Resources](#required-resources)
1213
- [Step 1: Ianvs Preparation](#step-1-ianvs-preparation)
1314
- [Step 2: Dataset and Model Preparation](#step-2-dataset-and-model-preparation)
1415
- [Dataset Configuration](#dataset-configuration)
@@ -69,7 +70,27 @@ To save API calls during multi-round testing, this example has designed a result
6970

7071
After all tests are completed, the Test Env Manager will calculate relevant metrics based on selected Metrics and hand over to Story Manager for printing test reports and generating Leader Board.
7172

72-
## Quick Start Guide for Benchmarking with Ianvs
73+
## Quick Start Guide
74+
75+
### Required Resources
76+
77+
Before using this example, you need to have the device ready:
78+
79+
- One machine is all you need, i.e., a laptop or a virtual machine is sufficient and a cluster is not necessary
80+
81+
- 2 CPUs or more
82+
83+
- 1 GPU with at least 6GB of memory, depends on the tested model
84+
85+
- 4GB+ free memory, depends on algorithm and simulation setting
86+
87+
- 10GB+ free disk space (depends on your model size)
88+
89+
- Internet connection for GitHub, PyPI, HuggingFace, etc
90+
91+
- Python 3.8+ environment
92+
93+
### Methods for Benchmarking with Ianvs
7394

7495
- To quickly experience benchmarking with Ianvs, proceed with the [Docker-Based Setup](#docker-based-setup).
7596
- For a detailed setup process, including creating a custom dataset, refer to the [Detailed Setup Guide](#detailed-setup-guide).
@@ -112,25 +133,7 @@ ianvs -f examples/cloud-edge-collaborative-inference-for-llm/benchmarkingjob.yam
112133

113134
- If you want to create a custom dataset, proceed to the next section.
114135

115-
## Detailed Setup Guide
116-
117-
### Required Resources
118-
119-
Before using this example, you need to have the device ready:
120-
121-
One machine is all you need, i.e., a laptop or a virtual machine is sufficient and a cluster is not necessary
122-
123-
- 2 CPUs or more
124-
125-
- 1 GPU with at least 6GB of memory, depends on the tested model
126-
127-
- 4GB+ free memory, depends on algorithm and simulation setting
128-
129-
- 10GB+ free disk space (depends on your model size)
130-
131-
- Internet connection for GitHub, PyPI, HuggingFace, etc
132-
133-
- Python 3.8+ environment
136+
### Detailed Setup Guide
134137

135138
#### Step 1. Ianvs Preparation
136139

@@ -318,7 +321,7 @@ Currently, supported routers include:
318321

319322
You can modify the `router` parameter in `test_queryrouting.yaml` to select the router you want to use.
320323

321-
For BERT router, you can use [routellm/bert](https://huggingface.co/routellm/bert) or [routellm/bert_mmlu_augmented](https://huggingface.co/routellm/bert_mmlu_augmented) or your own BERT model/
324+
For BERT router, you can use [routellm/bert](https://huggingface.co/routellm/bert) or [routellm/bert_mmlu_augmented](https://huggingface.co/routellm/bert_mmlu_augmented) or your own BERT model.
322325

323326
##### Data Processor Configuration
324327
The Data Processor allows you to custom your own data format after the dataset loaded.

0 commit comments

Comments
 (0)