Flux 2 dev, tp, mutated outputs

[putdanil]

When running Flux 2 Dev with Tensor Parallelism (TP=2) on 2×H100, generated images show double exposure / ghosting artifacts.
The issue is deterministic, affects img2img/edit strongly, also present in t2i, and happens only in certain environments.
I'm using flux 2 dev merged with turbo lora, it works absolutely fine on its own.

torchrun --nproc_per_node=2
-m cache_dit.serve.serve
--model-path flux2-dev-turbo-merged
--parallel-type tp
--parallel-text
--attn native
--generator-device cpu

Failing environment:

GPU: 2× NVIDIA H100 80GB

Driver: 570.172.08

CUDA runtime: 12.8

Torch: 2.10.0+cu128

NCCL: 2.27.5

diffusers: 0.36.0

transformers: 4.57.6

accelerate: 1.12.0

xformers: ❌

flash-attn: ❌

[putdanil]

I found out that this mutation only happens when you forward with TURBO_SIGMAS, without them its fine overall.

[DefTruth]

Hi~ @BBuf Could you please take a look at this issue when you have some free time? It seems related to serve module.