|
1 | | -# YANC_LMStudio |
| 1 | +# EA_LMStudio |
2 | 2 |
|
3 | | -A ComfyUI custom node for integrating LM Studio's local LLM/VLM inference. |
| 3 | +A ComfyUI custom node for local LLM/VLM inference via [LM Studio](https://lmstudio.ai/). |
4 | 4 |
|
5 | 5 | ## Features |
6 | 6 |
|
7 | | -- **Automatic Model Discovery**: Models are fetched from LM Studio at ComfyUI startup |
8 | | -- **Vision Model Support**: Up to 4 image inputs for VLM models (LLaVA, Qwen-VL, etc.) |
9 | | -- **Image Resizing**: Automatic resize options to speed up VLM inference |
10 | | -- **Reasoning Extraction**: Separate thinking/reasoning from final response |
11 | | -- **Speculative Decoding**: Optional draft model support |
12 | | -- **VRAM Management**: Unload models after use (enabled by default) |
| 7 | +- **Auto Model Discovery** - Models populate automatically from LM Studio at startup |
| 8 | +- **Vision Support** - Up to 4 image inputs with smart auto-resize to prevent OOM |
| 9 | +- **Reasoning Extraction** - Separates thinking from final response (DeepSeek R1, Qwen3, QwQ, etc.) |
| 10 | +- **Advanced Controls** - Temperature, top-k/p, repetition penalty, speculative decoding |
| 11 | +- **Smart Troubleshooting** - Helpful error messages with specific hints |
| 12 | +- **VRAM Management** - Auto-unload after generation (enabled by default) |
13 | 13 |
|
14 | 14 | ## Installation |
| 15 | + |
| 16 | +**Via ComfyUI Manager** (recommended): Search for "EA_LMStudio" |
| 17 | + |
| 18 | +**Manual:** |
15 | 19 | ```bash |
16 | 20 | cd ComfyUI/custom_nodes |
17 | | -git clone https://github.com/ALatentPlace/YANC_LMStudio.git |
18 | | -cd YANC_LMStudio |
19 | | -pip install -r requirements.txt |
| 21 | +git clone https://github.com/EnragedAntelope/EA_LMStudio.git |
| 22 | +pip install -r EA_LMStudio/requirements.txt |
20 | 23 | ``` |
21 | 24 |
|
22 | | -Or install via ComfyUI Manager. |
23 | | - |
24 | | -## Setup |
25 | | - |
26 | | -1. **Start LM Studio** with the server enabled (default: `http://127.0.0.1:1234`) |
27 | | -2. **Start ComfyUI** - models are auto-fetched at startup |
28 | | -3. Find the node under **YANC -> LMStudio** |
29 | | - |
30 | | -## Model Selection |
31 | | - |
32 | | -### Automatic (Recommended) |
33 | | -Models are fetched from LM Studio's `/v1/models` endpoint when ComfyUI starts. Select from the dropdown. |
34 | | - |
35 | | -**Requirement:** LM Studio must be running before starting ComfyUI. |
36 | | - |
37 | | -### Manual Entry |
38 | | -If LM Studio wasn't running at startup or you added models later: |
39 | | -1. Select `-- Custom (enter below) --` from dropdown |
40 | | -2. Enter model identifier in `custom_model_name` |
41 | | -3. Find identifiers in LM Studio's model list |
42 | | - |
43 | | -### Refresh Models |
44 | | -Enable `refresh_models`, queue once, then disable. Updates take effect on next node load. |
45 | | - |
46 | | -## Image Inputs (VLMs) |
47 | | - |
48 | | -The node supports up to 4 image inputs for vision-language models: |
49 | | -- **image1-4**: Optional image inputs (connect any combination) |
50 | | -- **image_resize**: Resize images before processing to speed up inference |
51 | | - - `No Resize` - Use original size |
52 | | - - `Low (512px)` - Fast processing |
53 | | - - `Medium (768px)` - Balanced (default) |
54 | | - - `High (1024px)` - Better detail |
55 | | - - `Ultra (1536px)` - Maximum detail |
56 | | - |
57 | | -**Note:** Not all VLMs support multiple images. If you get errors with multiple images, try using only `image1`. |
58 | | - |
59 | | -## Reasoning Extraction |
60 | | - |
61 | | -Many reasoning models (DeepSeek R1, Qwen3, QwQ, GLM-Z1) wrap their "thinking" process in special tags. The node can extract this separately from the final response. |
62 | | - |
63 | | -### Modes |
| 25 | +## Quick Start |
64 | 26 |
|
65 | | -- **Auto-detect (recommended)**: Automatically detects common reasoning patterns: |
66 | | - - `<think>...</think>` - DeepSeek R1, Qwen3, QwQ, GLM-Z1 |
67 | | - - `<thinking>...</thinking>` - Alternative format |
68 | | - - `<reasoning>...</reasoning>` - Some models |
69 | | - - GPT-OSS analysis channel format |
| 27 | +1. Start LM Studio with server enabled (default: `http://127.0.0.1:1234`) |
| 28 | +2. Start ComfyUI |
| 29 | +3. Find the node: **EA -> LMStudio** |
70 | 30 |
|
71 | | -- **Disabled**: Returns the full response as-is (no extraction) |
| 31 | +## Tips |
72 | 32 |
|
73 | | -- **Custom tags**: Specify your own open/close tags for models with unique formats |
| 33 | +- **Models not showing?** LM Studio must be running before ComfyUI starts. Use `refresh_models` checkbox to re-fetch. |
| 34 | +- **Context errors?** Increase context length in LM Studio settings (not max_tokens). |
| 35 | +- **VLM issues?** Try a smaller image resize option or single image if multi-image fails. |
| 36 | +- **Force thinking mode:** Add `/think` to prompts for Qwen3, or "Think step by step" for others. |
74 | 37 |
|
75 | | -### Output |
| 38 | +## Custom Server |
76 | 39 |
|
77 | | -- **response**: Final answer with reasoning tags removed (if extracted) |
78 | | -- **reasoning**: Extracted thinking/reasoning content |
79 | | - |
80 | | -This allows you to route reasoning to a separate display or log while keeping the final response clean. |
81 | | - |
82 | | -### Forcing Thinking Mode |
83 | | - |
84 | | -Models don't always produce thinking output for simple queries. To encourage thinking: |
85 | | -- **Qwen3 models**: Add `/think` to your prompt (e.g., "Explain quantum physics /think") |
86 | | -- **Other models**: Use phrases like "Think step by step" or "Show your reasoning" |
87 | | - |
88 | | -## Custom Server Address |
89 | | - |
90 | | -Default: `http://127.0.0.1:1234` |
91 | | - |
92 | | -To change, edit `lms_config/user_config.json`: |
| 40 | +Edit `lms_config/user_config.json`: |
93 | 41 | ```json |
94 | 42 | { |
95 | 43 | "server_host": "192.168.1.100", |
96 | | - "server_port": 1234, |
97 | | - "timeout_seconds": 5 |
| 44 | + "server_port": 1234 |
98 | 45 | } |
99 | 46 | ``` |
100 | | -This file survives git updates. |
101 | | - |
102 | | -## Parameters |
103 | | - |
104 | | -### Required |
105 | | -| Parameter | Default | Description | |
106 | | -|-----------|---------|-------------| |
107 | | -| system_message | "You are a helpful assistant." | System prompt defining LLM behavior | |
108 | | -| prompt | "" | User prompt / question | |
109 | | -| model_selection | - | Model dropdown or "Custom" | |
110 | | -| max_tokens | 1024 | Maximum OUTPUT tokens (see note below) | |
111 | | -| temperature | 0.7 | Randomness (0=deterministic, 1+=creative) | |
112 | | - |
113 | | -### Optional |
114 | | -| Parameter | Default | Description | |
115 | | -|-----------|---------|-------------| |
116 | | -| image_resize | Medium (768px) | Resize images before VLM processing | |
117 | | -| top_p | 1.0 | Nucleus sampling (lower=more focused) | |
118 | | -| top_k | 0 | Limits vocabulary (0=disabled, 20-40 recommended for thinking models) | |
119 | | -| repeat_penalty | 1.0 | Reduce repetition (1.1-1.3 recommended) | |
120 | | -| reasoning_mode | Auto-detect | How to extract reasoning from response | |
121 | | -| unload_llm | True | Unload LLM after generation (recommended) | |
122 | | - |
123 | | -### Understanding max_tokens vs Context Length |
124 | | - |
125 | | -- **max_tokens**: Limits OUTPUT tokens (how long the response can be) |
126 | | -- **Context Length**: Total tokens for INPUT + OUTPUT combined (set in LM Studio when loading model) |
127 | | - |
128 | | -If you see "Reached context length" errors, increase the model's context length in LM Studio's settings, not max_tokens. |
129 | 47 |
|
130 | 48 | ## Outputs |
131 | 49 |
|
132 | | -- **response**: The generated text |
133 | | -- **reasoning**: Extracted thinking/reasoning (if reasoning_tag found) |
134 | | -- **troubleshooting**: Status messages, errors, and hints |
| 50 | +| Output | Description | |
| 51 | +|--------|-------------| |
| 52 | +| response | Generated text (reasoning removed if extracted) | |
| 53 | +| reasoning | Extracted thinking content | |
| 54 | +| troubleshooting | Status messages and debug hints | |
135 | 55 |
|
136 | | -## Troubleshooting |
137 | | - |
138 | | -Check the `troubleshooting` output for detailed status information. |
| 56 | +## License |
139 | 57 |
|
140 | | -**Common Issues:** |
141 | | -- "Cannot connect": Ensure LM Studio server is running |
142 | | -- "Model not found": Verify model identifier matches LM Studio |
143 | | -- "Context length exceeded": Increase context length in LM Studio model settings |
144 | | -- Empty dropdown: Start LM Studio before ComfyUI, or use manual entry |
145 | | -- Multi-image errors: Model may only support single image, try just image1 |
| 58 | +[MIT License](LICENSE) |
146 | 59 |
|
147 | | -## License |
| 60 | +--- |
148 | 61 |
|
149 | | -MIT License |
| 62 | +*Based on [YANC_LMStudio](https://github.com/ALatentPlace/YANC_LMStudio) by A Latent Place* |
0 commit comments