Best prompt nodepack! Thanks. Ideas: amended instruction for 'MiniMax-H3 Reference Caption' (and instruction field in 'Multi Reference Caption')

[808charlie]

Now I've got things running, this really is great. Very configurable and flexible. Thank you!

I observed some models caption a picture well (albeit with an empty at the start of the description).
Other models at times return a ton of thinking (particularly Qwen 3.5, particularly variants of original model). They get to a great description, but think first.

The prompt you use for the Reference Caption node is output to the cli, so I'd already amended the prompt using a text input to the 'instruction' field in MiniMax-H3 Reference Caption. I then found adding "return only the prompt not your thinking" fixed the problem with 'thinking' in every case! I daresay other similar instructions to 'return only...' would have a similar effect.

The 'instruction' field for custom instructions is therefore great.
Request 1: any chance of an instruction field in the Multi Reference Caption node? Appreciate it might need one of pic/aud/vid as the prompt is not the same.

Request 2: Even more usefully, is it possible the 'MiniMax-H3 Prompt Writer' could include an 'instruction' field. You provide the 'system prompt' (instruction) exactly through the 'minimx-H3 guide promt (and LLM) node. Users could then edit it if needed, but importantly edit it for other models.

Why so much interest?

Best node pack I've found.

• Your nodes have significant advantage of others like 'ComfyUI-QwenVL' or 'ComfyUI_Qwen3VL-instruct' nodes, which require editing json files and changes to the node stop them trying to download models and to specify personally selected and pre-downloaded models. They use safetensors.
• MiniCPM node has above problems, but was one which used GGUF, but it had other issues as I've mentioned before.
• pixaroma has some prompt nodes, but whilst some of his are great, I have real issues with the slick functionality and JS (Claude coded) and how non-native they are and fragile the interaction feels - it feels like the difference between 'windows' and 'linux' :D
• Your nodes are faster than the native 'text-generate' that use safetensors and is much more fussy on models and much slower. Indeed, I've used safetensors models with yours, as you know, via clip-load, and for some reason it also takes WAY longer than loading a Q8 GGUF. Way longer. I suspect all the pinning and over-management of memory to be related.

Whilst I'd not normally be a fan of GGUF within ComfyUI (on my hardware), it really hits the spot for quick calls to LLMs doesn't it. Really impressed so far thanks.

[pytraveler]

It’s really great and nice how you describe my work. thank you very much. This is motivating.
Your requests have been accepted. I need a little time to make this more user‑friendly.
I’ll definitely let you know when I’m done.

[808charlie]

Thanks so much. I'll leave this one open. Sorry to have raised so many issues - kind of ironic that my lack-of-knowledge of python and understanding of llama.cpp has somehow influenced improvements to someone's code.

Would be great if a youtuber like benjiAI or nerdyrodent pick up on your nodes - that would really increase the visibility. The lightx2v loras almost seem a red-herring here, it's great you implemented them, but the more general prompt/img nodes are so welcome.
Get a break, no rush with the requests! Thank again.

[pytraveler]

Both are in 0.16.6, and so is the thing your workaround was working around.

The empty <think> </think> was mine

Not a quirk of those models. The CLIP captioner has always cut the thought block
out of an answer; the GGUF one never did, so it went straight into the caption -
and " ".join(caption.split()) is what turned <think>\n\n</think> into the
<think> </think> you were looking at. On a model that really reasons it is
worse than untidy: the whole deliberation lands in reference_assets, and from
there it travels on into the writer's prompt as if you had written it.

Both paths cut it now.

Cutting is only half of it, though, and the other half cannot be a flag. The
writer nodes render the chat template themselves and pass
enable_thinking=False. A caption cannot: llama-mtmd-cli applies the template
itself - that is what puts the media tokens in the right place in the turn - and
it publishes no switch for the thought channel. I checked the build here (b10599)
rather than guessing: no --chat-template-kwargs, no --reasoning-budget, only
--jinja. So the reasoning is still charged against --predict, which is how a
thinking model gives you a page of deliberation and half a sentence of caption.

Which is exactly what your sentence fixed. So it is now the last line of every
caption question, in the node rather than in your text field:

Do not think out loud: give the answer only, with no reasoning and no preamble.

It asks for no reasoning rather than for a description, which turned out to
matter: the first wording said "reply with the description only" and quietly
argued with every instruction that wanted something other than a description.

You found the right instrument. I have only bolted it on so nobody else has to.

Request 1: per reference, not per node

You anticipated the problem in the same breath as the request - one node, four
kinds of asset, and no single question that suits a picture, a clip and a sound
at once. So the field is not on the node. It is on the reference.

The strip of coloured squares that the Universal Writer has always had is now on
Multi Reference Caption too, and under each square there is a narrow band in the
same colour. Dark while that reference is asked its role's usual question, lit
once it is asked something else. Click it to write the question, right-click to
take it back, hover to read it. The text rides in a hidden JSON widget, so it
travels with the workflow and through the API like anything else.

The band has two lit states, and testing is what put them there. A checkbox in
the little editor decides which, and the band then says which it was:

• + instr - your line is added to the role's question.
• = instr, on a solid band - it is asked instead of it, and it takes over
  from length as well: whoever writes the question owns the shape of its
  answer.

Neither could be the only one. "Do not mention the window" is a rule, not a
question: asked on its own it leaves nothing wanting a description, and a 3B
model answers the rule - Subject 1: No. "Always answer 'blah blah blah'" is a
whole question, and leaving the role's in front of it hands the model a
contradiction, which it settles by describing anyway. Added is the default,
because a line typed into an empty box is usually the first kind.

Bringing the strip across had a second reason. The checkboxes on the input rows
were drawn straight onto the canvas, and Modern Node Design (Nodes 2.0) does
not draw the canvas - so that node quietly had no switches at all for anyone on
the new renderer. The squares are HTML and work in both.

Request 2: system_prompt on the writers

Yours is the better statement of it, so I will use it: the guide is the whole
of what makes these H3 writers. Nothing else in the node knows the format. So
system_prompt replaces the assembled guide outright, on the T2VA writer, the
Ref2VA writer and the Universal Writer - and when it holds anything, the guide is
not even fetched. That last part is for you specifically: an offline machine
should not have to download 24 KB of MiniMax's document in order to ignore it.

The way in is the one you spotted. MiniMax-H3 Guide Prompt (any LLM) hands
you the stock system prompt on an output; take it, edit it for Krea or LTX,
connect it back into system_prompt. Editing what the node would have sent
anyway beats writing one from nothing.

Two things do not move:

• The task message is never replaced. It carries the prompt, the aspect ratio
  and the duration, which every writing guide needs whatever house style it asks
  for - and it is the half you have no reason to rewrite.
• The answer is still split into the H3 sections. A guide that replies with a
  paragraph fills rewritten_prompt and leaves the section outputs empty. Worth
  knowing rather than worth avoiding.

One thing your report cost me, usefully

New widgets now go at the end of a node, always. ComfyUI restores a saved
node's widget values by position, so adding reference_instructions next to the
setting it belongs with - which is the obvious place - handed every widget below
it its neighbour's value. A workflow saved before the change came back with
max_frames holding what context_size had, and context_size holding a
boolean. Caught it here, nothing shipped in that state; but it is the sort of
thing that would have been very hard to report from the outside.

On the rest of what you wrote

Thank you, genuinely. Two of your points are worth answering rather than just
accepting.

Why the GGUF route is faster than a safetensors encoder here. It is not the
node being clever. CLIPLoader loads a full-precision text encoder through
ComfyUI's own allocator, with all the residency management that implies; the
GGUF route memory-maps a quantised file and hands it to a process that exits
afterwards. Different job, different cost. The CLIP path still wins when you have
several references in one run - it loads once for the lot, where the GGUF
captioner starts a process and reads the weights again for every asset.

The LoRAs are not a red herring, but you are right about the shape of it.
They are the reason the pack exists and the smallest part of what it now does:
everything guide-driven runs on any instruction-following GGUF, and after this
release it does not even have to be MiniMax's guide. That was worth someone
saying out loud.

Thanks for four reports in three days, every one of them precise enough to act
on. Two of them found real bugs and this one found a third.

[808charlie]

Ah, you do the right thing!
I've come across nodes where authors fiddled with widgets etc, and existing workflows didn't work, or field had 'NaN' etc. I tend to try and open new nodes rather than copy them around workflow to avoid it.

You found the right instrument. I have only bolted it on so nobody else has to.

Funnily enough it was the first thing I tried, without thinking, so that was lucky! I guess that's something to be said for natural language LLMs - you can speak to them in normal language without a lot of thought!

Thanks so much for all the changes. I'll grab the new release and try in a few days... need some non IT time... and have work to do now!

As it happens, I just recommended your node to someone who posted on ComfyUI issues with a llama cpp problem with a node they were using.

It might be worth creating a new custom node if the pack now extends beyond Minimax, as it's title might be broader?

Thanks again.

[pytraveler]

Good idea. And I’m just toying with the idea of reusing the nodes and, based on them, creating similar ones for other diffusion models.

[808charlie]

More to maintain that way? But equally a good idea to keep stuff split. Two times I'm frustrated by packs (repos) growing rather than separate nodes:

• RGthree - I had to fork the repo to make a 'PowerLTXLoraLoader' (the only item I have on github) to allow attenuation of 3 categories of audio layers (Audio, A2V and V2A) on LoRAs (kind of re-using Kijai's ideas but in the PowerLoraLoaderFormat). But I couldn't figure how to build it as stand alone node LIKE is PLL, as the node relies on lots of other pack functionality - so I had to just fork and add my crappy version. As it's a huge pack and he keeps making changes, I've not bothered to keep merging so I'll have to mark mine as stale - would have been much easier if PLL had been separate. I'm unlikely to bother with LTX given Minimax's superiority, so no interest in admin updating the fork. (I hope Kijai does a similar Lora node for minimax as he did for LTX). Seems odd to have a lora loader thrown in with critical things like mutter and context switches etc. That's a lot of work to pick apart if something happened like Nodes 2.0 got forced on us and RGthree stopped updating.

• Pixaroma - at an extreme, these nodes have some great ideas (like some of the cropping nodes etc) but has grown to include so much functionality, and numerous items which already exist, the latest being VRAM monitoring. Kijai had already bettered over the once-staple Crystools suite. Too much stuff in one repo.

So yea off that, perhaps it would be sensible to create other nodes for other models... but whether it's one huge repo with the lot or not will be interesting.

I'd best close this issue do you think?
You've included the functionality - thanks again - which warrant closure I guess. And I see you've added some sensible controls over image size etc; I can hardly keep up!

[pytraveler]

I completely agree with you. It’s better to share responsibility. Therefore, this project will be used exclusively for the purpose for which the code was developed. I will focus on usability and finding errors in the code.
Thank you again for your feedbacks! It was very helpful.