Skip to content

Commit 118ff92

Browse files
[Doc] Update V1 user guide for embedding and enc-dec models (vllm-project#19060)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
1 parent ec2dcd8 commit 118ff92

1 file changed

Lines changed: 10 additions & 5 deletions

File tree

docs/usage/v1_guide.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# vLLM V1
22

3+
**We have started the process of deprecating V0. Please read [RFC #18571](https://github.com/vllm-project/vllm/issues/18571) for more details.**
4+
35
V1 is now enabled by default for all supported use cases, and we will gradually enable it for every use case we plan to support. Please share any feedback on [GitHub](https://github.com/vllm-project/vllm) or in the [vLLM Slack](https://inviter.co/vllm-slack).
46

57
To disable V1, please set the environment variable as: `VLLM_USE_V1=0`, and send us a GitHub issue sharing the reason!
@@ -51,9 +53,9 @@ This living user guide outlines a few known **important changes and limitations*
5153
| **Spec Decode** | <nobr>🚧 WIP ([PR #13933](https://github.com/vllm-project/vllm/pull/13933))</nobr>|
5254
| **Prompt Logprobs with Prefix Caching** | <nobr>🟡 Planned ([RFC #13414](https://github.com/vllm-project/vllm/issues/13414))</nobr>|
5355
| **Structured Output Alternative Backends** | <nobr>🟡 Planned</nobr> |
54-
| **Embedding Models** | <nobr>🟡 Planned ([RFC #12249](https://github.com/vllm-project/vllm/issues/12249))</nobr> |
56+
| **Embedding Models** | <nobr>🚧 WIP ([PR #18015](https://github.com/vllm-project/vllm/pull/18015))</nobr> |
5557
| **Mamba Models** | <nobr>🟡 Planned</nobr> |
56-
| **Encoder-Decoder Models** | <nobr>🟡 Planned</nobr> |
58+
| **Encoder-Decoder Models** | <nobr>🟠 Delayed</nobr> |
5759
| **Request-level Structured Output Backend** | <nobr>🔴 Deprecated</nobr> |
5860
| **best_of** | <nobr>🔴 Deprecated ([RFC #13361](https://github.com/vllm-project/vllm/issues/13361))</nobr>|
5961
| **Per-Request Logits Processors** | <nobr>🔴 Deprecated ([RFC #13360](https://github.com/vllm-project/vllm/pull/13360))</nobr> |
@@ -63,10 +65,11 @@ This living user guide outlines a few known **important changes and limitations*
6365
- **🟢 Functional**: Fully operational, with ongoing optimizations.
6466
- **🚧 WIP**: Under active development.
6567
- **🟡 Planned**: Scheduled for future implementation (some may have open PRs/RFCs).
66-
- **🔴 Deprecated**: Not planned for v1 unless there is strong demand.
68+
- **🟠 Delayed**: Temporarily dropped in V1 but planned to be re-introduced later.
69+
- **🔴 Deprecated**: Not planned for V1 unless there is strong demand.
6770

6871
**Note**: vLLM V1’s unified scheduler treats both prompt and output tokens the same
69-
way by using a simple dictionary (e.g., {request_id: num_tokens}) to dynamically
72+
way by using a simple dictionary (e.g., `{request_id: num_tokens}`) to dynamically
7073
allocate a fixed token budget per request, enabling features like chunked prefills,
7174
prefix caching, and speculative decoding without a strict separation between prefill
7275
and decode phases.
@@ -140,7 +143,9 @@ vLLM V1 currently excludes model architectures with the `SupportsV0Only` protoco
140143
and the majority fall into the following categories. V1 support for these models will be added eventually.
141144

142145
**Embedding Models**
143-
Instead of having a separate model runner, hidden states processor [RFC #12249](https://github.com/vllm-project/vllm/issues/12249), which is based on global logits processor [RFC #13360](https://github.com/vllm-project/vllm/pull/13360), has been proposed to enable simultaneous generation and embedding using the same engine instance in V1. It is still in the planning stage.
146+
Initially, we will create a [separate model runner](https://github.com/vllm-project/vllm/pull/18015) to provide V1 support without conflicting with other ongoing work.
147+
148+
Later, we will consider using [hidden states processor](https://github.com/vllm-project/vllm/issues/12249), which is based on [global logits processor](https://github.com/vllm-project/vllm/pull/13360) to enable simultaneous generation and embedding using the same engine instance in V1. [PR #16188](https://github.com/vllm-project/vllm/pull/16188) is the first step towards enabling this.
144149

145150
**Mamba Models**
146151
Models using selective state-space mechanisms (instead of standard transformer attention)

0 commit comments

Comments
 (0)