"Save/cache images for this key" checkbox for key and a "clear image cache for key" action

[semenovnick]

Is this a feature relevant to companion itself, and not a specific module?

• I believe this to be a feature for companion, and is not specific to a module

Is there an existing issue for this?

• I have searched for similiar existing issues

Describe the feature

Good afternoon. In my projects, I use a large number of pages filled with key feedback (about 10 pages of 32*4 keys). I see, logically, a sharp increase in CPU usage each time feedback occurs, caused in part by re-rendering key images (as far as I know, only JPEGs are sent to Streamdeck). As far as I know, there is no more or less standard (Nvidia) GPU acceleration for encoding JPEGs. In my projects, and perhaps in many complex ones, the number of truly unique key images is limited (excluding dynamic text), and my suggestion is to add a "Save/cache images for this key" checkbox and a "clear image cache for key" action with the key path as the parameter. which will reduce the load on the CPU (call the render once when unique key properties appear, which are stored, say, in the form of a hash of these properties)

Usecases

Prevent re render already rendered images for reducing CPU usage. Manual image cache control

[Julusian]

I don' t think that leaving this up to the user to manage is the correct thing to do, instead we should be trying to get companion to handle it better automatically.

From my measurements, the jpeg encoding is really quick to do (and will be hard to cache in a predictable way). I suspect it is the general bitmap generation that is taking the time.
We are supposed to have a LRU cache on that to avoid constant redraws of the same thing, but that appears to have broken a long time ago.
So I have fixed that, and adjusted the size, so that it hopefully behaves a bit better for larger configs.
Hopefully that is enough to resolve your issues. Its in the latest beta, are you able to give that a try to see?

[semenovnick]

I tested the new beta version. Under identical conditions, I saw a performance boost of approximately 30% with the beta version, which is an excellent result.

Version 4.2.2

Beta version

Here is test page with feedbacks in my test. Every button has 4 (defined in $(custom:someVar)) different states switching every second, and optional to switch red feedback every 5 second blinks

MANYMANY_FEEDBACKS.zip

This is close to my workflow in terms of the number of simultaneously updated key images: each key has 4-5 fixed states (conventionally: enabled, disabled, on, off, positive feedback)

Due to the fact that these states are indeed fixed for a long period of time (during a live broadcast session), there is a need to be able to manage caching in order to minimize the load during a live broadcast (cache possible feedback in advance before the live broadcast during pre-broadcast preparation)

And in my feature request, I didn't mean to completely switch control of this cache to manual mode, but rather to provide the ability to switch to this manual mode in some kind of advanced mode, so to speak...

Overall, we really don't want to abandon your wonderful product. But the fact is, we're starting to hit the performance limits of the PCs running Companion (even with the optimizations performed within Companion).

[semenovnick]

Perhaps, if manual cache management isn't possible, there might be a way to implement feedback that would directly use key images without the rendering step?

For example something like Direct image feedback? Then I could move caching outside Companion...

[dnmeid]

If you are dealing with performance and a lot of buttons drawing, another hint is to use fixed font sizes instead of "auto". Calculating the correct font size is a very time consuming process for a button redraw.

[Julusian]

Perhaps, if manual cache management isn't possible, there might be a way to implement feedback that would directly use key images without the rendering step?

The challenge here is that during most of the rendering stage we don't know what model of streamdeck it will be on (or if it is on a streamdeck) and we also need a render to show in the ui. Until we know what streamdeck model, we dont know what dimension of jpeg it wants, or what rotation it needs to be applied to the jpeg. I think some do some non-standard color order too, and older models use bmp instead of jpeg (companion currently is not aware of these, and leaves them to the streamdeck library to handle)
I am open to pulling some of that encoding back into companion to allow it to be more optimal, but going back to before the rendering will be very complex and will intertwine the areas quite tightly.
And thats ignoring satellite; which pushes pngs over the network for satellite to translate to the correct thing needed per streamdeck.

So maybe some improvements can be done, but I don't think bypassing our drawing is it.

---

Due to the fact that these states are indeed fixed for a long period of time (during a live broadcast session), there is a need to be able to manage caching in order to minimize the load during a live broadcast (cache possible feedback in advance before the live broadcast during pre-broadcast preparation)

I suppose I could take our caching in a different direction; there is currently a global pool of caching, (which can be good if multiple buttons are doing the same thing), maybe making the size of this more configurable would be good. I don't want to set it too high as that means more memory usage, but there could be a setting to increase it for those who are ok with that cost. Maybe the new formula isnt agressive enough either.

Or another approach is maybe that global pool is not the right approach. Maybe it should be caching per button, and then per button we could have that option to choose a cache size. I am worried that will mean that more will be cached for everyone though; for it to cache anything for anyone it will need to cache 4 or more images (assuming 1 feedback that toggles between 2 states, + pressed and unpressed states). It would probably need a ttl to try and keep it sensible, but it could still get large. (maybe purging the cache each time the style options are edited?)
While today the size of this cache is fairly predictable, with the graphics overhaul, each cache item will likely be at elast 200kb, easily more (it will depend on what surfaces it gets used by)

I am also a little worried about the cost of how we compute cache keys, stringifying the whole drawing object won't always be cheap (it could contain multiple mb of images). Maybe this can be improved, in case it is a noticable cost..

The optimal solution might be a mix of the two, perhaps with the per button cache set to a size of 0 by default.
And of course all of this is guesswork, and it could be that its something outside of this that is being the slow part, that wont be noticable without some profiling..

Maybe part of this is we use a fixed number of renderers of these images too, currently set to 6. Maybe increasing that for you would help too

Calculating the correct font size is a very time consuming process for a button redraw.

Good point, I guess time to tackle that too

[semenovnick]

What if a disk-backed cache was used in addition to the in-memory one? Something along these lines:

Request
   ↓
Quick-LRU (finalKey → encoded image)
   ↓
Disk (finalKey → encoded image)
   ↓
Master cache (masterHash → RGBA / vector)
   ↓
Layout renderer (canvas / svg)
   ↓
Encoder (sharp → webp/png)

Where the cache keys could be split into two parts, whrere content is stringified parameters of knob (text, fontsize, aligment etc...):

masterHash = sha256(content)
variantKey = "256x128:jpg:q80" // for different surfaces
finalKey   = masterHash + ":" + variantKey

And on disk it could look something like:

cache/
  a9/
    a912bc34/
        72x72.jpg
        40x30.jpg
        60x60.png

Memory (Quick-LRU) would keep only the hot encoded variants, while disk could act as a much larger, slower cache for warm data.

On modern machines, NVMe/SSD access is often fast enough that a large disk cache can significantly reduce re-renders without requiring huge amounts of RAM. For setups with mostly static configurations (e.g. predefined layouts and broadcasts in studios), this might be especially useful.

Not sure if this aligns with the project’s goals, but I thought it might be worth considering as an optional or future optimization.