Skip to content

Commit c0767d8

Browse files
committed
Minor fixes. Changes to table of contents and required resources moved up
Minor fixes. Changes to table of contents and required resources moved up Signed-off-by: Aryan <nandaaryan823@gmail.com> grammar corrected and table of contents changed Signed-off-by: Aryan <nandaaryan823@gmail.com>
1 parent 7dd12c2 commit c0767d8

1 file changed

Lines changed: 26 additions & 23 deletions

File tree

  • examples/cloud-edge-collaborative-inference-for-llm

examples/cloud-edge-collaborative-inference-for-llm/README.md

Lines changed: 26 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,11 @@
55
- [Why LLM Need Cloud-Edge Collaborative Inference?](#why-llm-need-cloud-edge-collaborative-inference)
66
- [Possible Collaborative Inference Strategy](#possible-collaborative-inference-strategy)
77
- [Details of Design](#details-of-design)
8-
- [Quick Start Guide for Benchmarking with Ianvs](#quick-start-guide-for-benchmarking-with-ianvs)
8+
- [Quick Start Guide](#quick-start-guide)
9+
- [Required Resources](#required-resources)
10+
- [Methods for benchmarking with ianvs](#methods-for-benchmarking-with-ianvs)
911
- [Docker-Based Setup](#docker-based-setup)
1012
- [Detailed Setup Guide](#detailed-setup-guide)
11-
- [Required Resources](#required-resources)
1213
- [Step 1: Ianvs Preparation](#step-1-ianvs-preparation)
1314
- [Step 2: Dataset and Model Preparation](#step-2-dataset-and-model-preparation)
1415
- [Dataset Configuration](#dataset-configuration)
@@ -72,7 +73,27 @@ To save API calls during multi-round testing, this example has designed a result
7273

7374
After all tests are completed, the Test Env Manager will calculate relevant metrics based on selected Metrics and hand over to Story Manager for printing test reports and generating Leader Board.
7475

75-
## Quick Start Guide for Benchmarking with Ianvs
76+
## Quick Start Guide
77+
78+
### Required Resources
79+
80+
Before using this example, you need to have the device ready:
81+
82+
- One machine is all you need, i.e., a laptop or a virtual machine is sufficient and a cluster is not necessary
83+
84+
- 2 CPUs or more
85+
86+
- 1 GPU with at least 6GB of memory, depends on the tested model
87+
88+
- 4GB+ free memory, depends on algorithm and simulation setting
89+
90+
- 10GB+ free disk space (depends on your model size)
91+
92+
- Internet connection for GitHub, PyPI, HuggingFace, etc
93+
94+
- Python 3.8+ environment
95+
96+
### Methods for Benchmarking with Ianvs
7697

7798
- To quickly experience benchmarking with Ianvs, proceed with the [Docker-Based Setup](#docker-based-setup).
7899
- For a detailed setup process, including creating a custom dataset, refer to the [Detailed Setup Guide](#detailed-setup-guide).
@@ -115,25 +136,7 @@ ianvs -f examples/cloud-edge-collaborative-inference-for-llm/benchmarkingjob.yam
115136

116137
- If you want to create a custom dataset, proceed to the next section.
117138

118-
## Detailed Setup Guide
119-
120-
### Required Resources
121-
122-
Before using this example, you need to have the device ready:
123-
124-
One machine is all you need, i.e., a laptop or a virtual machine is sufficient and a cluster is not necessary
125-
126-
- 2 CPUs or more
127-
128-
- 1 GPU with at least 6GB of memory, depends on the tested model
129-
130-
- 4GB+ free memory, depends on algorithm and simulation setting
131-
132-
- 10GB+ free disk space (depends on your model size)
133-
134-
- Internet connection for GitHub, PyPI, HuggingFace, etc
135-
136-
- Python 3.8+ environment
139+
### Detailed Setup Guide
137140

138141
#### Step 1. Ianvs Preparation
139142

@@ -315,7 +318,7 @@ Currently, supported routers include:
315318

316319
You can modify the `router` parameter in `test_queryrouting.yaml` to select the router you want to use.
317320

318-
For BERT router, you can use [routellm/bert](https://huggingface.co/routellm/bert) or [routellm/bert_mmlu_augmented](https://huggingface.co/routellm/bert_mmlu_augmented) or your own BERT model/
321+
For BERT router, you can use [routellm/bert](https://huggingface.co/routellm/bert) or [routellm/bert_mmlu_augmented](https://huggingface.co/routellm/bert_mmlu_augmented) or your own BERT model.
319322

320323
##### Data Processor Configuration
321324
The Data Processor allows you to custom your own data format after the dataset loaded.

0 commit comments

Comments
 (0)