Skip to content

Commit 0d480c8

Browse files
author
Eric Zhong
committed
added comments after using skill
1 parent f32e060 commit 0d480c8

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

.github/config/model-tests/vllm-multimodal-benchmark-tests.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,12 @@ benchmark:
2828
test_script: "vllm_embedding_benchmark_test.sh"
2929
min_rps: 3
3030

31+
# LightOnOCR-2-1B: 1B VLM for document OCR (LightOnOcrForConditionalGeneration).
32+
# bf16 ~2GB, fits L40S easily. Thresholds are provisional — calibrate after first run.
3133
- name: "lighton-ocr-2-1b"
3234
s3_model: "lightonocr-2-1b.tar.gz"
3335
runner_label: "gpu-l40s-1gpu-runners"
34-
extra_args: "--dtype bfloat16 --max-model-len 16384"
36+
extra_args: "--dtype bfloat16 --max-model-len 4096"
3537
min_throughput: 30
3638
min_rps: 1
3739
benchmark_profiles: "baseline,high_concurrency"

0 commit comments

Comments
 (0)