Comfy is sooo much faster at higher resoluting

[Volnovik]

Looks like kontext is properly layered and comfyui implementation has some smart offloading going in the background.
I gave it a shot at upscaling with bf16 kontext to 2048x2656 and it was surprisingly fast. Unfortunately it has no Mixture of Diffusers implementation for flux, and then I saw that you saved the day again and implemented it in Forge. Thanks btw.
Yet on comfy I get 3.77s/it and on Forge I get 8.97s/it with my 4090. Sage+triton installed for both. Swap to shared, tried various gpu memory quotas - this was maximum that I've got. Any idea why 2 times difference in inference speed?

[DenOfEquity]

Two possible options, given that you've already tried the most likely one of adjusting the GPU Weights to avoid hitting shared memory:

1. if using to output sizing in the extension you would be using larger reference images than if using the scaling node in Comfy. Potentially also if using no change. Larger references = slower.
2. I'm not sure how effective SageAttention is in Forge (my GPU is too old to use it), based on add sage-attention and flash-attention support,add support for noob ctrlnet models,automaticly add local themes in tmp/gradio_themes lllyasviel/stable-diffusion-webui-forge#2881, and the older PR linked within, performance seems variable.

There is some code in the extension to automatically reserve extra VRAM, but I'm not sure how well it works in practice compared to adjusting GPU Weights, so it is commented out (lines 162, 212, 247-249).