Skip to content

v2.0.0: remove two no-op widgets, add streaming/interrupt, structured output, honest diagnostics - #19

Merged
EnragedAntelope merged 2 commits into
mainfrom
v2.0.0-audit
Aug 9, 2026
Merged

v2.0.0: remove two no-op widgets, add streaming/interrupt, structured output, honest diagnostics#19
EnragedAntelope merged 2 commits into
mainfrom
v2.0.0-audit

Conversation

@EnragedAntelope

Copy link
Copy Markdown
Owner

Why

The lmstudio SDK silently discards prediction-config keys it does not recognise instead of raising. v1.x relied on that not happening, and shipped two widgets that never did anything:

Widget Sent as Reality
presence_penalty presencePenalty not in LlmPredictionConfigDict — dropped
enable_thinking enableThinking not in LlmPredictionConfigDict — dropped

Confirmed against a live LM Studio server: the prediction_config the server echoes back contains neither. LM Studio has no presence penalty, no frequency penalty, no inference-time seed and no thinking on/off flagseed exists only in the load config.

To stop this recurring, generate() now diffs the config it sent against the config the server reports as applied, and warns about anything dropped.

Breaking change, mitigated

Both widgets are removed and the rest regrouped (sampling → output shaping → reasoning → vision → speculative decoding → management). ComfyUI serialises widgets_values positionally, so this would shift every later value — web/ea_lmstudio.js detects the v1.5.x layout and writes values onto current widgets by name, which is order-independent.

Browser-verified with a real v1 workflow: without the migration reasoning_mode loads as "[[/R]]" and custom_open_tag as false; with it, all 23 legacy values land correctly on the regrouped widgets and the four new ones take their defaults.

Fixes

  • A successful generation could be reported as Generation failed. Every LlmPredictionStats field except stop_reason is Optional; formatting None with :.2f raised TypeError after the model had produced its text, which the outer handler turned into an error and discarded.
  • Restored workflows drew widgets outside the node frame. A workflow stores the node's size and LiteGraph restores it verbatim without re-checking that the widgets fit. v2.0.0 has a net +2 widgets, so every upgraded workflow was too short. growToFitWidgets grows (never shrinks) the node — a v1 node stored at 700px now grows to 820px on load.
  • New nodes defaulted to the -- Custom -- sentinel with an empty identifier, so the first run always failed even when discovery worked.
  • Models whose identifier fails validation (LM Studio serves ids like model@?) vanished from the dropdown with no explanation.
  • requires-python said >=3.9; the lmstudio dependency itself needs >=3.10.

Added

  • Streamed prediction — ComfyUI's cancel button actually stops generation, and the queue progress bar tracks tokens.
  • Native reasoning split — uses LM Studio's own reasoning_type fragment tagging when available; tag regexes remain the fallback.
  • stop_strings, context_overflow, and structured JSON output (schema-constrained, with automatic ```json fence recovery).
  • In-node response previewOUTPUT_NODE now carries a ui payload. Excluded from both the API prompt and the saved workflow.
  • Speculative-decoding acceptance stats plus a low-acceptance hint.
  • example_workflows/ (discoverable in ComfyUI's template browser), AGENTS.md, and CI running the suite on 3.10 and 3.13.

Verification

  • 102 tests pass (was 48).
  • Live QA against LM Studio 0.4.x + SDK 1.5.0: text, VLM, stop strings, structured output, invalid schema, context policy, all sampling params, mid-stream cancel.
  • Browser-verified: node registration, v1 migration onto regrouped widgets, in-node preview, serialization, widget/node overflow and node overlap in both example workflows, no console errors from this pack.

🤖 Generated with Claude Code

EnragedAntelope and others added 2 commits August 9, 2026 12:20
… output, honest diagnostics

The lmstudio SDK silently discards prediction-config keys it does not
recognise instead of raising. v1.x relied on that not happening, and shipped
two widgets that never did anything:

  * presence_penalty -> sent as presencePenalty
  * enable_thinking  -> sent as enableThinking

Neither key exists in LlmPredictionConfigDict, so both were dropped before the
request left the machine. Confirmed against a live server: the prediction_config
LM Studio echoes back contains neither. LM Studio has no presence penalty, no
frequency penalty, no inference-time seed and no thinking on/off flag.

Both widgets are removed. Because ComfyUI serialises widgets_values
positionally, removing two mid-list widgets would shift every later value, so
web/ea_lmstudio.js detects the v1.5.x layout and realigns by widget name on
load (verified in a browser: without it reasoning_mode loads as "[[/R]]").

To stop this class of bug recurring, generate() now diffs the config it sent
against the config the server reports as applied and warns about anything
dropped.

Fixes
  * A successful generation could be reported as "Generation failed":
    LlmPredictionStats fields are Optional and formatting None with :.2f raised
    TypeError after the model had already produced its text.
  * New nodes defaulted to the "Custom" sentinel with an empty identifier, so
    the first run always failed even when discovery worked.
  * Models whose identifier fails validation (LM Studio serves ids like
    "model@?") vanished from the dropdown with no explanation.
  * requires-python said >=3.9; the lmstudio dependency needs >=3.10.

Added
  * Streamed prediction: ComfyUI's cancel button now actually stops generation,
    and the queue progress bar tracks tokens.
  * Reasoning is taken from LM Studio's own reasoning_type fragment tagging
    when available, with the tag regexes kept as fallback.
  * stop_strings, context_overflow, and structured JSON output (schema-
    constrained, with automatic ```json fence recovery).
  * OUTPUT_NODE now carries a ui payload: the response renders in the node.
  * Speculative-decoding acceptance stats and a low-acceptance hint.
  * example_workflows/ (discoverable via ComfyUI's template browser), AGENTS.md,
    and a CI workflow running the suite on 3.10 and 3.13.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t layout debt

Layout was incoherent: image_resize sat between seed and the draft model, two
empty textareas broke up the sampling rows, and json_schema showed even in Text
mode. Widgets are now grouped most-used first:

  prompts -> model -> generation -> sampling -> output shaping -> reasoning
  -> vision -> speculative decoding -> management

Reordering is safe precisely now: no v2 workflow exists yet, and the v1
migration writes values onto current widgets by name, so it is order
independent. Verified in a browser - a v1 workflow lands correctly on the
regrouped widgets, including image_resize and custom_draft_model.

Also fixes a real defect this surfaced: a workflow stores the node's size and
LiteGraph restores it verbatim without re-checking that the widgets still fit.
v2.0.0 has a net two more widgets than v1.5.x, so every upgraded workflow was
restored too short and the overflow drew outside the node frame.
growToFitWidgets grows (never shrinks) the node on configure and after
execution; a v1 node stored at 700px now grows to 820px on load.

For the record: a stock ComfyUI Note node's textarea overhangs its own frame by
~13 units at any size. That is upstream behaviour, measured against a freshly
created Note, and is not something this pack causes or should chase.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@EnragedAntelope
EnragedAntelope merged commit ebc047f into main Aug 9, 2026
2 checks passed
@EnragedAntelope
EnragedAntelope deleted the v2.0.0-audit branch August 9, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant