Optimize loading images from memory

[CarbonPool]

I have an image stream, and the image is continuously updated during the convenient process, but the image is delayed, and the reading and updating process is time-consuming. Is there any way to optimize it?

[CarbonPool]

let rgb_buf = image::load_from_memory(&file.as_bytes()).unwrap().into_rgb8(); let shared_buf: SharedPixelBuffer<Rgb8Pixel> = SharedPixelBuffer::clone_from_slice(rgb_buf.as_raw(), rgb_buf.width(), rgb_buf.height()); let image = slint::Image::from_rgb8(shared_buf); app.set_image(image);

[tronical]

Hi! I am not entirely sure that I understand the question correctly - apologies if I’m off here :). However, I suggest that of loading the file from disk is expensive (because of I/O or decoding), then you could consider doing that in a separate thread with the image crate. When you’re ready, load the pixel data into a SharedPixelBuffer, call upgrade_in_event_loop on your weak app handle, and - in the closure - create the slint::Image from the pixel buffer and finally set the property. That way the operations in the UI thread are limited to a little bit of reference counting, instead of file I/O and decoding.

Does that make sense? I wonder if we need an example perhaps that does that by loading an image feed from the network. Not sure what would be a good feed though.

[CarbonPool]

I use reqwest to download the http multipart/x-mixed-replace mjpeg stream image and update the Image, but each frame passes through image::load_from_memory and creates a SharedPixelBuffer. On the low-performance iot development board, there will be a certain performance consumption, resulting in a screen Delay, if slint::Image:load_from_data is possible, may be a good solution

[tronical]

If we had slint::Image:load_from_data, then it would use image::load_from_memory behind the scenes though. Meanwhile, you can do the call to image::load_from_memory and creation of the SharedPixelBuffer in a separate thread and only do the last step in the gui thread.

So to answer the original question then: I'm not sure there's a way to optimise image loading further :(.

I'll convert this to a discussion and we can continue to explore use-cases. It would be interesting to know what performance limits you're hitting. Is it that you board becomes noticeably slower even if you do the decoding in a thread? Would it make sense to explore a different image format that's faster to decode? (what format are you using?)

[CarbonPool]

I use 720p mjpeg image, I think cloning and creating SharedPixelBuffer is unnecessary

[tronical]

slint::Image internally uses a SharedPixelBuffer. SharedPixelBuffer is very cheap to clone because it is shared. I think creating a SharedPixelBuffer outside and initialising a slint::Image from it is very cheap.

[tronical]

I'll convert this to a discussion. Let's explore more there :)