Skip to content

Commit 82c3446

Browse files
authored
[release branch] bfcl demo fixes (#3871)
### 🛠 Summary #3833 #3834
1 parent 5203745 commit 82c3446

File tree

2 files changed

+6
-6
lines changed

2 files changed

+6
-6
lines changed

demos/continuous_batching/accuracy/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -109,22 +109,22 @@ openai_compatible (model_version=OpenGVLab/InternVL2_5-8B,max_retries=1), gen_kw
109109
Use [Berkeley function call leaderboard ](https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard)
110110

111111

112-
```bash
112+
```text
113113
git clone https://github.com/ShishirPatil/gorilla
114114
cd gorilla/berkeley-function-call-leaderboard
115115
git checkout cd9429ccf3d4d04156affe883c495b3b047e6b64
116116
curl -s https://raw.githubusercontent.com/openvinotoolkit/model_server/refs/heads/releases/2025/4/demos/continuous_batching/accuracy/gorilla.patch | git apply -v
117117
pip install -e .
118118
```
119119
The commands below assumes the models is deployed with the name `ovms-model`. It must match the name set in the `bfcl_eval/constants/model_config.py`.
120-
```bash
120+
```text
121121
export OPENAI_BASE_URL=http://localhost:8000/v3
122122
bfcl generate --model ovms-model --test-category simple,multiple --temperature 0.0 --num-threads 100 -o --result-dir model_name_dir
123123
bfcl evaluate --model ovms-model --result-dir model_name_dir
124124
```
125125

126126
Alternatively, use the model name `ovms-model-stream` to run the tests with stream requests. The results should be the same.
127-
```bash
127+
```text
128128
export OPENAI_BASE_URL=http://localhost:8000/v3
129129
bfcl generate --model ovms-model-stream --test-category simple,multiple --temperature 0.0 --num-threads 100 -o --result-dir model_name_dir
130130
bfcl evaluate --model ovms-model-stream --result-dir model_name_dir

demos/continuous_batching/accuracy/gorilla.patch

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
diff --git a/berkeley-function-call-leaderboard/bfcl_eval/constants/model_config.py b/berkeley-function-call-leaderboard/bfcl_eval/constants/model_config.py
2-
index 73731c0..0d966ed 100644
2+
index 73731c0..b6bbf48 100644
33
--- a/berkeley-function-call-leaderboard/bfcl_eval/constants/model_config.py
44
+++ b/berkeley-function-call-leaderboard/bfcl_eval/constants/model_config.py
55
@@ -2060,6 +2060,30 @@ third_party_inference_model_map = {
@@ -18,9 +18,9 @@ index 73731c0..0d966ed 100644
1818
+ is_fc_model=True,
1919
+ underscore_to_dot=True,
2020
+ ),
21-
+ "ovms_model_stream": ModelConfig(
21+
+ "ovms-model-stream": ModelConfig(
2222
+ model_name="ovms-model-stream",
23-
+ display_name="ovms_model_stream",
23+
+ display_name="ovms-model-stream",
2424
+ url="http://localhost:8000/v3",
2525
+ org="ovms",
2626
+ license="apache-2.0",

0 commit comments

Comments
 (0)