You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/usage/v1_guide.md
+10-5Lines changed: 10 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,7 @@
1
1
# vLLM V1
2
2
3
+
**We have started the process of deprecating V0. Please read [RFC #18571](https://github.com/vllm-project/vllm/issues/18571) for more details.**
4
+
3
5
V1 is now enabled by default for all supported use cases, and we will gradually enable it for every use case we plan to support. Please share any feedback on [GitHub](https://github.com/vllm-project/vllm) or in the [vLLM Slack](https://inviter.co/vllm-slack).
4
6
5
7
To disable V1, please set the environment variable as: `VLLM_USE_V1=0`, and send us a GitHub issue sharing the reason!
@@ -51,9 +53,9 @@ This living user guide outlines a few known **important changes and limitations*
@@ -63,10 +65,11 @@ This living user guide outlines a few known **important changes and limitations*
63
65
-**🟢 Functional**: Fully operational, with ongoing optimizations.
64
66
-**🚧 WIP**: Under active development.
65
67
-**🟡 Planned**: Scheduled for future implementation (some may have open PRs/RFCs).
66
-
-**🔴 Deprecated**: Not planned for v1 unless there is strong demand.
68
+
-**🟠 Delayed**: Temporarily dropped in V1 but planned to be re-introduced later.
69
+
-**🔴 Deprecated**: Not planned for V1 unless there is strong demand.
67
70
68
71
**Note**: vLLM V1’s unified scheduler treats both prompt and output tokens the same
69
-
way by using a simple dictionary (e.g., {request_id: num_tokens}) to dynamically
72
+
way by using a simple dictionary (e.g., `{request_id: num_tokens}`) to dynamically
70
73
allocate a fixed token budget per request, enabling features like chunked prefills,
71
74
prefix caching, and speculative decoding without a strict separation between prefill
72
75
and decode phases.
@@ -140,7 +143,9 @@ vLLM V1 currently excludes model architectures with the `SupportsV0Only` protoco
140
143
and the majority fall into the following categories. V1 support for these models will be added eventually.
141
144
142
145
**Embedding Models**
143
-
Instead of having a separate model runner, hidden states processor [RFC #12249](https://github.com/vllm-project/vllm/issues/12249), which is based on global logits processor [RFC #13360](https://github.com/vllm-project/vllm/pull/13360), has been proposed to enable simultaneous generation and embedding using the same engine instance in V1. It is still in the planning stage.
146
+
Initially, we will create a [separate model runner](https://github.com/vllm-project/vllm/pull/18015) to provide V1 support without conflicting with other ongoing work.
147
+
148
+
Later, we will consider using [hidden states processor](https://github.com/vllm-project/vllm/issues/12249), which is based on [global logits processor](https://github.com/vllm-project/vllm/pull/13360) to enable simultaneous generation and embedding using the same engine instance in V1. [PR #16188](https://github.com/vllm-project/vllm/pull/16188) is the first step towards enabling this.
144
149
145
150
**Mamba Models**
146
151
Models using selective state-space mechanisms (instead of standard transformer attention)
0 commit comments