Multi-backend pool, async-completion hardening, LLM-friendly responses, 42-workflow library

[svilendotorg]

This is an umbrella PR for ~20 patches accumulated in svilendotorg/ComfyUI-MCP-Server over a homelab deployment. Full per-patch write-up + rationale lives in IMPROVEMENTS.md. I'm happy to split into smaller PRs for any subset you want to land — just say which ones and I'll cherry-pick onto fresh branches.

Headline features

Multi-backend pool with VRAM-aware routing — comfyui_pool.py (new)

Drop-in replacement for ComfyUIClient that fronts multiple ComfyUI instances over COMFYUI_URLS=name=url,.... Routing modes:

• Default: least-queue across reachable backends; ties broken by env order. Backends with foreign_vram > POOL_MAX_FOREIGN_VRAM_GB (default 4 GB) are excluded — distinguishes "ComfyUI's own warm checkpoint" (allowed, fast) from "another process is squatting" (skip).
• Explicit: every workflow tool now accepts a backend="..." parameter that overrides the filter.

VRAM signal sourced from a small companion custom_node (ComfyUI-ForeignVRAM) that exposes /foreign_vram[?max_foreign_gb=N] (returns 503 when over threshold so an upstream Traefik / NGINX healthcheck can also kick busy backends out of LB rotation).

Async-completion asset registration — tools/job.py

Workflows over the 30s sync timeout return a running handle. Polling get_job(prompt_id) previously returned outputs but never wrote to AssetRegistry, so list_assets / view_image couldn't find them. Now get_job walks completed-prompt outputs and registers each via AssetRegistry.register_asset. Pool stamps _pool_backend_url onto the returned history dict so the registration uses the correct hostname.

LLM-friendly response shape

Every workflow + get_job response now includes markdown_preview — a ready-to-paste markdown snippet. Image MIMEs → ![alt](url) (Claude Desktop renders inline images in chat). Audio → [🔊 filename](url). Video → [🎬 ...]. 3D → [🎲 ...]. Tool descriptions are augmented with a hint to paste the field.

Optional topic slug + auto-derive

Every workflow tool accepts an optional topic arg (1–2 word slug). Server slugifies and injects between the date and workflow id in filename_prefix:

images/2026-05-03-giraffe_picture_sdxl_t2i_00001_.png

When the LLM skips topic, the server auto-derives one — first from prompt/tags (drop English stopwords + generic words like realistic, image); failing that, parse the source image filename and carry the slug forward (so sdxl_t2i → hunyuan3d_i2mesh chains keep one coherent topic).

Friendlier image / audio inputs

image, image_last, and audio parameters now accept any of:

• a full URL (existing — auto-uploaded to ComfyUI's input/)
• an asset_id returned by a previous tool call (resolved via the registry)
• a bare output filename (resolved via get_asset_by_identity scan over known output subfolders)

Lets the LLM chain workflows naturally without manually constructing URLs.

Mesh / 3D output keys

Output-key matching previously knew only images / image / gifs / gif / audio / audios / files. Hunyuan3D workflows emit a 3d key — files landed on disk but the MCP raised an exception and the LLM never saw the asset. Adds MESH_OUTPUT_KEYS = ("3d", "mesh", "glb", "gltf", "obj") and extends _guess_output_preferences to detect mesh-class workflows.

upload_file MCP tool — tools/upload.py (new)

A new tool for pushing arbitrary binary files (images / audio / video) into ComfyUI's input folder via base64. Single-shot for ≤ 100 KB; chunked mode (chunk_index + total_chunks + upload_id) for larger files (≤ 100 MB total) — MCP tool calls don't handle multi-MB base64 payloads gracefully otherwise.

Workflow library — 42 workflows + embedded _meta

42 workflow JSONs covering image / audio / video / 3D mesh tasks. Each has a top-level _meta key with timing / resolution / best-for / param hints; WorkflowManager._load_workflow_metadata reads from there and uses it as the MCP tool description. The _meta key is stripped before submission to ComfyUI's /prompt endpoint.

Naming: {model}_{task}[_{variant}] — tasks include t2i i2i t2v i2v d2v fl2v t2a t2mv i2mesh a2v. Namespace (image / video / audio / mesh) auto-detected from the task suffix.

Output subfolder convention

Every workflow's filename_prefix writes into a typed subfolder of output/ — images/ for *_t2i* / *_i2i*, audio/ for *_t2a*, video/ for *_t2v* / *_i2v* / *_fl2v* / *_d2v* / sonic_a2v, mesh/ for hunyuan3d_*.

Misc fixes

• Substitute %date:yyyy-MM-dd% patterns in filename_prefix at submit time (ComfyUI's SaveImage doesn't do it natively).
• Translate model paths between Windows (backslash) and Linux (forward slash) so the same workflow runs on either backend.
• Pool stamps result["backend_url"] = client.base_url after submit so asset URLs returned to the caller point at the actual backend hostname, not the registry's localhost default.
• A pile of small JSON corrections: EmptyFlux2LatentImage / EmptyLTXVLatentVideo batch_size copy-paste bugs (set to 1), ResizeImageMaskNode DynamicCombo dot-prefix sub-fields, ComfyMathExpression values.a / values.b re-binding, removed ClownSampler paths from LTX-2.3 workflows, PARAM_AUDIO injection for LoadAudio nodes, second LoadImage node in wan22_fl2v correctly bound to PARAM_IMAGE_LAST.

Splitting

If any subset is welcome and others aren't, I can rebase individual commits onto separate branches and open them as standalone PRs. Just say which ones.

Tested

Two RTX 3090s on one host (LXC, dual-instance ComfyUI) + an RTX 4090 on a separate Windows host, fronted by Traefik LB and the multi-backend pool. All 42 workflows run end-to-end across all three backends.

[svilendotorg]

@joenorton