Skip to content

Commit ae70cbb

Browse files
committed
Update docs/communication.mddocs/communication.md.
Signed-off-by: Revital Sur <eres@il.ibm.com>
1 parent 2de19de commit ae70cbb

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

docs/communication.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,9 @@ For image 0 (given `token_ids[0]=1` as BOS, `token_ids[1]=32000` as placeholder
446446
}
447447
```
448448

449+
> [!NOTE]
450+
> Each `kwargs_data.image[i]` may be `null` (and the whole `kwargs_data` field may be absent) when the image is already resident in the multimodal cache under `mm_hashes.image[i]`. A `null` item carries no pixel tensor: the worker resolves that image from the cache by hash instead of deserializing a tensor (`token_in_token_out/serving.py`, "None -> cache hit"). The coordinator emits `null` for any entry whose `KwargsData` is empty; a `null` still occupies a slot, so the per-image slice lengths still match. The example above shows the tensor-carrying (cache-miss) form.
451+
449452
`model` is required by the `/inference/v1/generate` request validator and must match the served model name. Coordinator sources it from `reqCtx.Model` (populated from the inbound request body's `model` field).
450453

451454
#### Response

0 commit comments

Comments
 (0)