[Prototype] Add interactive deployment config generator#318
[Prototype] Add interactive deployment config generator#318
Conversation
Add a reusable interactive configuration selector widget (similar to SGLang Cookbook's ConfigGenerator) that lets users pick hardware platform, quantization, tensor parallelism, and other options to auto-generate the correct vllm serve command. Includes a proof-of-concept integration in the Llama 3.3 70B recipe with options for Blackwell/Hopper hardware, NVFP4/FP8 quantization, TP size, and prefix caching. Other recipes can adopt the widget by adding a small HTML/JS block that defines model-specific options and a generateCommand function. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces an interactive command generator for Llama 3.3-70B deployment, allowing users to select hardware, quantization, and parallelism settings to generate a vLLM serve command. The implementation includes a new vanilla JavaScript utility, custom CSS, and updates to the documentation and mkdocs configuration. Review feedback focuses on ensuring the generated command correctly overrides YAML defaults for prefix caching, optimizing the initialization logic in the JavaScript class, and adding safety checks for the clipboard API.
| if (values.prefixCaching === 'disabled') { | ||
| cmd += ' \\\n --no-enable-prefix-caching'; | ||
| } |
There was a problem hiding this comment.
The current logic only adds the --no-enable-prefix-caching flag when "Disabled" is selected. However, the provided YAML configuration files (Llama3.3_Blackwell.yaml and Llama3.3_Hopper.yaml) already have no-enable-prefix-caching: true set. This means that selecting "Enabled" in the interactive tool will not actually enable prefix caching, as the YAML setting will persist. To ensure the interactive tool works as expected, you should explicitly add the --enable-prefix-caching flag when "Enabled" is selected, as CLI arguments typically override configuration file settings in vLLM.
| if (values.prefixCaching === 'disabled') { | |
| cmd += ' \\\n --no-enable-prefix-caching'; | |
| } | |
| if (values.prefixCaching === 'enabled') { | |
| cmd += ' \\\n --enable-prefix-caching'; | |
| } else { | |
| cmd += ' \\\n --no-enable-prefix-caching'; | |
| } |
| _getInitialState() { | ||
| const state = {}; | ||
| for (const [key, option] of Object.entries(this.config.options)) { | ||
| let items = option.items; | ||
| if (option.getDynamicItems) { | ||
| // Bootstrap: build default values from static items first | ||
| const defaults = {}; | ||
| for (const [k, opt] of Object.entries(this.config.options)) { | ||
| if (opt.items && opt.items.length > 0) { | ||
| const d = opt.items.find(i => i.default); | ||
| defaults[k] = d ? d.id : opt.items[0].id; | ||
| } | ||
| } | ||
| items = option.getDynamicItems(defaults); | ||
| } | ||
| const defaultItem = items && items.find(i => i.default); | ||
| state[key] = defaultItem ? defaultItem.id : (items && items[0] ? items[0].id : ''); | ||
| } | ||
| return state; | ||
| } |
There was a problem hiding this comment.
The defaults object is currently recalculated inside the loop for every option that uses getDynamicItems. It is more efficient to pre-calculate these defaults once at the beginning of the method.
| _getInitialState() { | |
| const state = {}; | |
| for (const [key, option] of Object.entries(this.config.options)) { | |
| let items = option.items; | |
| if (option.getDynamicItems) { | |
| // Bootstrap: build default values from static items first | |
| const defaults = {}; | |
| for (const [k, opt] of Object.entries(this.config.options)) { | |
| if (opt.items && opt.items.length > 0) { | |
| const d = opt.items.find(i => i.default); | |
| defaults[k] = d ? d.id : opt.items[0].id; | |
| } | |
| } | |
| items = option.getDynamicItems(defaults); | |
| } | |
| const defaultItem = items && items.find(i => i.default); | |
| state[key] = defaultItem ? defaultItem.id : (items && items[0] ? items[0].id : ''); | |
| } | |
| return state; | |
| } | |
| _getInitialState() { | |
| const state = {}; | |
| const defaults = {}; | |
| for (const [k, opt] of Object.entries(this.config.options)) { | |
| if (opt.items && opt.items.length > 0) { | |
| const d = opt.items.find(i => i.default); | |
| defaults[k] = d ? d.id : opt.items[0].id; | |
| } | |
| } | |
| for (const [key, option] of Object.entries(this.config.options)) { | |
| let items = option.items; | |
| if (option.getDynamicItems) { | |
| items = option.getDynamicItems(defaults); | |
| } | |
| const defaultItem = items && items.find(i => i.default); | |
| state[key] = defaultItem ? defaultItem.id : (items && items[0] ? items[0].id : ''); | |
| } | |
| return state; | |
| } |
| copyBtn.addEventListener('click', () => { | ||
| navigator.clipboard.writeText(command).then(() => { | ||
| copyBtn.textContent = 'Copied!'; | ||
| setTimeout(() => { copyBtn.textContent = 'Copy'; }, 1500); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Accessing navigator.clipboard without checking for its existence can lead to runtime errors in non-secure contexts (HTTP) or older browsers where the API is not available.
| copyBtn.addEventListener('click', () => { | |
| navigator.clipboard.writeText(command).then(() => { | |
| copyBtn.textContent = 'Copied!'; | |
| setTimeout(() => { copyBtn.textContent = 'Copy'; }, 1500); | |
| }); | |
| }); | |
| copyBtn.addEventListener('click', () => { | |
| if (!navigator.clipboard) return; | |
| navigator.clipboard.writeText(command).then(() => { | |
| copyBtn.textContent = 'Copied!'; | |
| setTimeout(() => { copyBtn.textContent = 'Copy'; }, 1500); | |
| }); | |
| }); |
Summary
How to add to other recipes
Recipe authors just add a
<div>and<script>block (~40 lines) defining model-specific options and agenerateCommandfunction. The base component (assets/config-generator.js+assets/config-generator.css) handles all the rendering and state management.Test plan
mkdocs serveand verify the Llama 3.3 70B recipe shows the interactive selector🤖 Generated with Claude Code