cc @asomoza

Flux doesn't have the behavior of the the other models because we have the distilled version of the model, so this is just my speculation but, for the other inpainting/tile controlnets, you must have the control_guidance_end to 1.0 or you'll get a different image at the end which defeats the purpose of a pure inpainting controlnet.

Of course you can still play with the end result if you want a somewhat similar image, but that also can be achieved with plain img2img or you can do a mix of techniques to achieve a good results so having control_guidance_start and control_guidance_start would be a nice thing to have.

I'm not sure it'd help Flux to be good at inpainting though, so better if you keep you expectations low about this. Maybe you can test it with ComfyUI if it has it and post the results?

In my case I'm not using inpainting conventionnaly to inpaint small areas and add or modify elements. (Like the dog on the bench example)

I'm using it to modify 90% of the image, and preserve stuff like like outdoor view from an interior photo.
And then I add ControlNets to keep the structure of the room intact with Depth or MLSD etc. and transform the style of the image. (Example in this video: https://www.youtube.com/watch?v=3E5V27jLHxU)

I saw a picture of a ComfyUI workflow on Twitter showing start_percent: 0.000 and end_percent: 0.3000 which I assume is the equivalent to control_guidance_start and end ? And I see the result is really good (except it doesn't use inpainting in the example to prevent modifying the window).

That's what led me to think those options might be helpful in my case..

start_percent: 0.000 and end_percent: 0.3000 which I assume is the equivalent to control_guidance_start and end ?

you're correct, and yes, if you don't care about the final result and use the image as a guide, you can use the control_guidance_end.

If you look at the image though, the controlnet isn't doing that much , you can probably achieve the same result with img2img and prompting. The only similarity with the control image is that there's a room, a window and a bed, it's missing a whole window and the bookshelf, and the shapes aren't even the same for the other ones.

When you say 'preserve the outdoor view' of the scene, you mean 100%? If you lower the guidance end that won't be the result though.

Just so you know, I'm not against this feature, just giving my opinion. If no one else from the community does it, I can allocate some time soon and implement this feature and test it. I use this option a lot with SDXL so I can see the benefit of it.

When you say 'preserve the outdoor view' of the scene, you mean 100%? If you lower the guidance end that won't be the result though.

Yes 100%. However the rest of the room will be modified with a strength between 0.7 and 1.

My goal is too use a highcontrolnet_conditioning_scale like between 0.6 and 0.8. But at the same time a control_guidance_end that ends pretty soon like 0.3 or 0.4. So that the base structure of the output respects well the different ControlNets, and the rest of the diffusion process is more "free" so that Flux can do its magic without control.

I think that's worth trying honestly, Diffusers library is much more developer friendly than ComfyUI, and with Flux being very "strict" with its parameters range when combined with ControlNet + Inpainting, I think it may help reach more consistent results.

If you need examples, here are the input, mask and output I usually get With SD 1.5 with some ControlNets:

With Flux I'm nowhere near that result yet.

alright, I added the labels and let's see if someone from the community is willing to do this, if not, when I have the time I'll implement it.

Thanks a lot @asomoza 🙏

@asomoza I have added the control_guidance_start and end for all three flux controlnet files.