Skip to content

Add FaceDetailer tiled VAE controls and optional post_detail_shrink#1195

Open
xmarre wants to merge 61 commits into
ltdrdata:Mainfrom
xmarre:Main
Open

Add FaceDetailer tiled VAE controls and optional post_detail_shrink#1195
xmarre wants to merge 61 commits into
ltdrdata:Mainfrom
xmarre:Main

Conversation

@xmarre

@xmarre xmarre commented Mar 12, 2026

Copy link
Copy Markdown

Summary

This PR now has two main parts:

  1. expand FaceDetailer’s VAE tiling support so the tiled path is respected consistently, can auto-adapt by crop size, and exposes explicit tile controls
  2. add an optional post_detail_shrink correction for the higher-resolution patch-size drift I observed with FLUX.2 Klein 9B

There is also a small hardening fix for Impact Pack’s existing final resize path after detail decode.

FaceDetailer tiled VAE changes

This branch broadens FaceDetailer’s tiled VAE support in a few ways.

A. FaceDetailer now respects tiled VAE encode more consistently

The branch fixes the path where FaceDetailer/inpaint conditioning could still bypass the intended tiled VAE encode behavior.

To do that, it introduces a small proxy VAE wrapper for the inpaint-conditioning path so encode requests still go through tiled encode with the resolved tile settings when that path is selected.

B. Adaptive tiled encode can now stay on the tiled path

The branch adds adaptive tile-size resolution based on crop resolution:

  • >= 3.0 MP -> 128 tiles
  • >= 1.5 MP -> 256 tiles
  • otherwise -> 512 tiles

with overlap defaulting to max(16, tile_size // 8) unless the user supplies an explicit overlap.

This matters because the problem I was hitting was not just “use tiled VAE once”. FaceDetailer needed to stay on the tiled encode path consistently, including the 512/64 case, instead of dropping back to the plain VAE path for those smaller adaptive tile sizes.

C. FaceDetailer nodes now expose explicit VAE tile controls

The relevant FaceDetailer nodes now expose:

  • force_adaptive_tiled_encode
  • tile_size
  • tile_overlap

These are threaded through the FaceDetailer / FaceDetailerPipe path into encode/decode.

In addition, the existing tiled decode path now uses the resolved tile size / overlap instead of always hardcoding the old default decode settings, while still guarding the overlap kwarg for older ComfyUI versions.

Optional post_detail_shrink

This PR also keeps the original opt-in post_detail_shrink feature.

I added it to compensate for a behavior I observed with FLUX.2 Klein 9B: when FaceDetailer regenerates crops above roughly ~1 MP, the returned patch can come back slightly and fairly uniformly enlarged relative to the original surrounding image.

Because FaceDetailer only regenerates the cropped face region and then composites that result back into the untouched source image, that size drift becomes visible at the seam.

This feature does not try to solve the root cause inside FLUX.2 Klein 9B. It adds a practical, opt-in correction on the Impact Pack side that shrinks the regenerated patch slightly before compositing it back.

Added controls:

  • post_detail_shrink (BOOLEAN, default: False)
  • post_detail_shrink_scale (FLOAT, default: 0.995, step: 0.001)

When enabled:

  • the regenerated patch is resized down slightly after the detail pass
  • the resized patch is anchored around the detected bbox center
  • the paste mask is shifted/scaled together with the regenerated patch
  • the cropped seg mask is shifted/scaled as well so downstream mask and SEGS consumers stay aligned

This uses the new helper:

  • utils.shift_within_canvas(...)

Mask output fix

FaceDetailer.enhance_face() now builds its output mask from the updated returned segs after detailing instead of always rebuilding it from the original input segs.

Without that change, enabling post_detail_shrink could leave the public mask output larger than the actually corrected region.

Resize-path hardening

Independently of the tiling and shrink changes, I also hit a crash in Impact Pack’s existing final resize path after detail decoding while running a FLUX.2 FaceDetailer workflow.

The crash stack ended in:

  • PIL.Image.resize
  • modules/impact/utils.py:tensor_resize
  • modules/impact/core.py:enhance_detail

To harden that path, this branch also makes two small changes:

  • modules/impact/core.py:enhance_detail() now only calls utils.tensor_resize(...) when the decoded image size actually differs from the requested (w, h)
  • modules/impact/utils.py:tensor_resize(...) now normalizes w / h to plain integers, rejects non-positive targets, and returns early for same-size requests instead of entering Pillow unnecessarily

Scope / compatibility notes

The post_detail_shrink feature remains intentionally scoped around the FLUX.2 Klein 9B behavior I observed. It is disabled by default.

The tiled VAE work is broader in applicability than post_detail_shrink, because it improves how FaceDetailer handles larger crops and how it preserves the intended tiled encode/decode path.

One important compatibility note: this branch intentionally changes the default FaceDetailer encode behavior by adding force_adaptive_tiled_encode = True on the FaceDetailer node path. That means FaceDetailer will prefer the adaptive tiled encode path by default. Users who want the older fallback behavior can disable force_adaptive_tiled_encode.

So this PR is not purely a no-behavior-change refactor. The shrink feature is opt-in, but the FaceDetailer tiled-encode behavior is intentionally strengthened by default.

Practical effect

For users, the main outcomes are:

  • FaceDetailer is less likely to fall back to the plain VAE encode path when the intent is to stay tiled
  • FaceDetailer now exposes tile-size / overlap controls directly
  • tiled decode uses the same resolved tile geometry instead of fixed defaults
  • FLUX.2 users get an optional seam-reduction correction via post_detail_shrink
  • the existing resize path is harder to crash on invalid or no-op resize requests

xmarre and others added 6 commits March 12, 2026 06:15
### Motivation

- Provide an optional post-detail shrink so refined patches can be slightly reduced around the detected bbox center before being pasted back, avoiding over-expanded detail artifacts.
- Expose a fine-grained user control for the shrink amount with sensible defaults (`0.995`, step `0.001`) to match observed working cases.
- Add a small helper to reliably shift/compose masks and tensors when a refined patch is resized and offset back into its original crop canvas.

### Description

- Add `_apply_post_detail_shrink(...)` to `modules/impact/impact_pack.py` which rescales the refined patch and mask around the bbox center, computes a bbox-centered offset, pastes the shrunken image into a copy of the original crop, and returns the shifted paste mask and cropped mask.
- Thread two new node inputs `post_detail_shrink` (`BOOLEAN`) and `post_detail_shrink_scale` (`FLOAT`, default `0.995`, step `0.001`) through the detailer entry points and forwarding calls across the Detailer/AutoRetry/Pipe/Face variants so the setting is available and propagated.
- Use the shrink-adjusted mask (`cropped_mask_for_output`) when constructing `SEG` outputs so downstream consumers receive the shifted/resized mask corresponding to the shrunken patch.
- Add `shift_within_canvas(...)` to `modules/impact/utils.py`, a dtype/device-preserving helper that shifts 2D/3D/4D numpy or torch arrays into a canvas with a fill value and returns the same type (torch tensors preserve device/dtype).

### Testing

- Ran `python -m py_compile modules/impact/impact_pack.py modules/impact/utils.py` and the files compiled successfully.
- Cleaned generated caches with `find modules/impact -type d -name __pycache__ -prune -exec rm -rf {} +` to ensure no stale bytecode remained.
- Verified that the core module was not altered and inspected the diffs of the modified files to confirm only the intended changes were added (no syntax errors detected).
Add optional post-detail shrink and canvas shift utility for Detailer nodes
…tailer

Use post-detail segments for FaceDetailer mask generation
@xmarre

xmarre commented Mar 16, 2026

Copy link
Copy Markdown
Author

I pushed a small hardening update for Impact Pack’s existing final resize path after detail decoding.

This is separate from the optional post_detail_shrink feature.

The crash stack ended in:

  • PIL.Image.resize
  • modules/impact/utils.py:tensor_resize
  • modules/impact/core.py:enhance_detail

So this update now:

  • skips the final tensor_resize(...) call when the decoded image is already at the requested (w, h)
  • normalizes resize targets to plain ints, rejects non-positive targets, and returns early for same-size requests before entering Pillow

After this change, the same workflow and same seed that had triggered the resize-path crash completed successfully in my testing.

I also updated the PR body so the current branch scope is documented accurately.

@xmarre xmarre changed the title Add optional post_detail_shrink to FaceDetailer Add FaceDetailer tiled VAE controls and optional post_detail_shrink Apr 17, 2026
xmarre and others added 30 commits May 20, 2026 20:02
…freeze-fix

[codex] Fix FaceDetailer resize quality regression
…ty-shrink-freeze-fix

Revert "[codex] Fix FaceDetailer resize quality regression"
…-inpaint-size-guard

Revert "[codex] Guard force inpaint working size"
Avoid RGB resize for tiny post-detail shrink
Reduce FaceDetailer frame cleanup churn
[codex] Disable hot path detailer cleanup
Move detailer latents to CPU before VAE decode
…guard

Guard WSL SAM release before detail segments
[codex] Guard detailer paste against non-finite crops
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant