Add FaceDetailer tiled VAE controls and optional post_detail_shrink#1195
Open
xmarre wants to merge 61 commits into
Open
Add FaceDetailer tiled VAE controls and optional post_detail_shrink#1195xmarre wants to merge 61 commits into
xmarre wants to merge 61 commits into
Conversation
### Motivation
- Provide an optional post-detail shrink so refined patches can be slightly reduced around the detected bbox center before being pasted back, avoiding over-expanded detail artifacts.
- Expose a fine-grained user control for the shrink amount with sensible defaults (`0.995`, step `0.001`) to match observed working cases.
- Add a small helper to reliably shift/compose masks and tensors when a refined patch is resized and offset back into its original crop canvas.
### Description
- Add `_apply_post_detail_shrink(...)` to `modules/impact/impact_pack.py` which rescales the refined patch and mask around the bbox center, computes a bbox-centered offset, pastes the shrunken image into a copy of the original crop, and returns the shifted paste mask and cropped mask.
- Thread two new node inputs `post_detail_shrink` (`BOOLEAN`) and `post_detail_shrink_scale` (`FLOAT`, default `0.995`, step `0.001`) through the detailer entry points and forwarding calls across the Detailer/AutoRetry/Pipe/Face variants so the setting is available and propagated.
- Use the shrink-adjusted mask (`cropped_mask_for_output`) when constructing `SEG` outputs so downstream consumers receive the shifted/resized mask corresponding to the shrunken patch.
- Add `shift_within_canvas(...)` to `modules/impact/utils.py`, a dtype/device-preserving helper that shifts 2D/3D/4D numpy or torch arrays into a canvas with a fill value and returns the same type (torch tensors preserve device/dtype).
### Testing
- Ran `python -m py_compile modules/impact/impact_pack.py modules/impact/utils.py` and the files compiled successfully.
- Cleaned generated caches with `find modules/impact -type d -name __pycache__ -prune -exec rm -rf {} +` to ensure no stale bytecode remained.
- Verified that the core module was not altered and inspected the diffs of the modified files to confirm only the intended changes were added (no syntax errors detected).
Add optional post-detail shrink and canvas shift utility for Detailer nodes
…tailer Use post-detail segments for FaceDetailer mask generation
Author
|
I pushed a small hardening update for Impact Pack’s existing final resize path after detail decoding. This is separate from the optional The crash stack ended in:
So this update now:
After this change, the same workflow and same seed that had triggered the resize-path crash completed successfully in my testing. I also updated the PR body so the current branch scope is documented accurately. |
Hand off local changes and respect tiled VAE encode
FaceDetailer: auto-switch to adaptive tiled VAE encode for large crops
Fix FaceDetailer fallback to plain VAE at adaptive 512 tiles
Fixed 3 file(s) based on 3 unresolved review comments. Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
Fixed 2 file(s) based on 2 unresolved review comments. Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
Add FaceDetailer VAE tile size/overlap controls
[codex] Mitigate FaceDetailer CUDA freezes
[codex] Avoid PIL in tensor resize
…ze-fix [codex] Fix FaceDetailer post-decode paste-back path
…freeze-fix [codex] Fix FaceDetailer resize quality regression
…ty-shrink-freeze-fix Revert "[codex] Fix FaceDetailer resize quality regression"
…-inpaint-size-guard Revert "[codex] Guard force inpaint working size"
Avoid RGB resize for tiny post-detail shrink
Reduce FaceDetailer frame cleanup churn
[codex] Disable hot path detailer cleanup
Move detailer latents to CPU before VAE decode
…guard Guard WSL SAM release before detail segments
Guard enhance_detail entry blur path
[codex] Use OpenCV for CPU tensor resize
Use OpenCV for WSL mask blur
[codex] Guard detailer paste against non-finite crops
[codex] Fix linked wildcard UI feedback
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR now has two main parts:
post_detail_shrinkcorrection for the higher-resolution patch-size drift I observed with FLUX.2 Klein 9BThere is also a small hardening fix for Impact Pack’s existing final resize path after detail decode.
FaceDetailer tiled VAE changes
This branch broadens FaceDetailer’s tiled VAE support in a few ways.
A. FaceDetailer now respects tiled VAE encode more consistently
The branch fixes the path where FaceDetailer/inpaint conditioning could still bypass the intended tiled VAE encode behavior.
To do that, it introduces a small proxy VAE wrapper for the inpaint-conditioning path so encode requests still go through tiled encode with the resolved tile settings when that path is selected.
B. Adaptive tiled encode can now stay on the tiled path
The branch adds adaptive tile-size resolution based on crop resolution:
>= 3.0 MP->128tiles>= 1.5 MP->256tiles512tileswith overlap defaulting to
max(16, tile_size // 8)unless the user supplies an explicit overlap.This matters because the problem I was hitting was not just “use tiled VAE once”. FaceDetailer needed to stay on the tiled encode path consistently, including the
512/64case, instead of dropping back to the plain VAE path for those smaller adaptive tile sizes.C. FaceDetailer nodes now expose explicit VAE tile controls
The relevant FaceDetailer nodes now expose:
force_adaptive_tiled_encodetile_sizetile_overlapThese are threaded through the FaceDetailer / FaceDetailerPipe path into encode/decode.
In addition, the existing tiled decode path now uses the resolved tile size / overlap instead of always hardcoding the old default decode settings, while still guarding the
overlapkwarg for older ComfyUI versions.Optional post_detail_shrink
This PR also keeps the original opt-in
post_detail_shrinkfeature.I added it to compensate for a behavior I observed with FLUX.2 Klein 9B: when FaceDetailer regenerates crops above roughly ~1 MP, the returned patch can come back slightly and fairly uniformly enlarged relative to the original surrounding image.
Because FaceDetailer only regenerates the cropped face region and then composites that result back into the untouched source image, that size drift becomes visible at the seam.
This feature does not try to solve the root cause inside FLUX.2 Klein 9B. It adds a practical, opt-in correction on the Impact Pack side that shrinks the regenerated patch slightly before compositing it back.
Added controls:
post_detail_shrink(BOOLEAN, default:False)post_detail_shrink_scale(FLOAT, default:0.995, step:0.001)When enabled:
This uses the new helper:
utils.shift_within_canvas(...)Mask output fix
FaceDetailer.enhance_face()now builds its output mask from the updated returned segs after detailing instead of always rebuilding it from the original input segs.Without that change, enabling
post_detail_shrinkcould leave the publicmaskoutput larger than the actually corrected region.Resize-path hardening
Independently of the tiling and shrink changes, I also hit a crash in Impact Pack’s existing final resize path after detail decoding while running a FLUX.2 FaceDetailer workflow.
The crash stack ended in:
PIL.Image.resizemodules/impact/utils.py:tensor_resizemodules/impact/core.py:enhance_detailTo harden that path, this branch also makes two small changes:
modules/impact/core.py:enhance_detail()now only callsutils.tensor_resize(...)when the decoded image size actually differs from the requested(w, h)modules/impact/utils.py:tensor_resize(...)now normalizesw/hto plain integers, rejects non-positive targets, and returns early for same-size requests instead of entering Pillow unnecessarilyScope / compatibility notes
The
post_detail_shrinkfeature remains intentionally scoped around the FLUX.2 Klein 9B behavior I observed. It is disabled by default.The tiled VAE work is broader in applicability than
post_detail_shrink, because it improves how FaceDetailer handles larger crops and how it preserves the intended tiled encode/decode path.One important compatibility note: this branch intentionally changes the default FaceDetailer encode behavior by adding
force_adaptive_tiled_encode = Trueon the FaceDetailer node path. That means FaceDetailer will prefer the adaptive tiled encode path by default. Users who want the older fallback behavior can disableforce_adaptive_tiled_encode.So this PR is not purely a no-behavior-change refactor. The shrink feature is opt-in, but the FaceDetailer tiled-encode behavior is intentionally strengthened by default.
Practical effect
For users, the main outcomes are:
post_detail_shrink