Support loading resources at runtime through callback

[ubruhin]

Feature Description

I'm a bit surprised I could not find an issue about this yet, so here I'm creating it:

Currently (for a C++ application) basically we have two options for resource loading (images, fonts):

• as-absolute-path: Loaded at runtime by Slint, with absolute path
• embed-files: Embedded into the application

Where as-absolute-path is a no-go as it will not work when deploying the application to another computer. So embed-files is the only real option. But this also comes with problems:

• Bloats the compilation and the application binary
• No advanced control over embedding, e.g. using compression (to allow developers to choose between file size vs. speed)
• Does not allow the rest of the application to access the same resources, forcing the developers to bundle the same files multiple times in the same application
• Does not allow to deploy the resources as files (e.g. in /usr/local/share/librepcb/myfont.ttf)

Let me explain it from the LibrePCB point of view:

• The TTF fonts used in the Slint UI are also used for other parts of the application, in particular for QGraphicsScene and various Qt dialogs/widgets. We already have a proven setup/worflow for deploying the TTF fonts and loading them at runtime, but we can't reuse this for Slint as the fonts are getting embedded.
• The same also applies to images, which are used by both the Slint UI and other parts of the application (e.g. Qt widgets and some parts of the backend). For this, we use Qt's resource system, so we have already bundled those images in our application. But we can't make Slint using those images, so we have to bundle them twice.

What I really miss is the ability to not bundle any files by Slint, but to load them at runtime through a callback which we can register. Slint should not care about where those files are coming from - in our case, TTF fonts would come from the file system, but images would come from the bundled Qt resources.

I am thinking about something like setting DEFAULT_SLINT_EMBED_RESOURCES=from-user-callback, and some setters like slint::set_load_image_callback(...) and slint::set_load_font_callback(). The callbacks would get a file path relative to appwindow.slint and return a binary blob (Vec<u8> or whatever).

One could argue it would be cleaner to provide only a single callback for any kind of resource. But honestly that would make our live a bit hard as we would have to parse the passed file path to determine where it needs to be loaded from (fonts from file system, images from Qt resources). But in worst case, I think that would be possible (still better than the current solution).

Actually I even wonder whether it's good to have only one DEFAULT_SLINT_EMBED_RESOURCES for any kind of resources, or whether it would be worth to control embedding separately for every kind of resource. I could imagine for example that it might be desired to load fonts at runtime, but let Slint embed images.

Curious what you think about this.

Product Impact

No response

[tronical]

This is a good topic to discuss and it's also complex in the sense that there are many ways of approaching "resource".

(And I have strong opinions on the topic ;)

What I really miss is the ability to not bundle any files by Slint, but to load them at runtime through a callback which we can register.

What I could envision is a mode where the application supplies a path at run-time to a directory that mirrors a hierarchy available at compile time, so that the compiler verify the existence of files and their validity, but instead of embedding the data in the executable it is expected to be deployed separately (up to the app) and Slint is informed at run-time.

Would that work for you?

[ubruhin]

This is a good topic to discuss and it's also complex in the sense that there are many ways of approaching "resource".

I agree :) I can imagine different applications have different needs how to approach this topic.

Would that work for you?

If I understand your proposal correctly: It would work for fonts as we deploy them to a share directory which was also used at compile time. But it wouldn't work for images because we would like to load them from the Qt resources system.

For images, we actually use a bit of a complex concept. We have included Git submodules for both, Bootstrap Icons and Font Awesome in our repository. From Slint, we access any icons like this:

source: @image-url("../../../font-awesome/svgs/solid/folder-open.svg");
source: @image-url("../../../bootstrap-icons/icons/floppy-fill.svg");

Some of the icons are also used by other parts of the application, so we add them to the *.qrc file. There we even use aliases to avoid the ugly, long filepaths:

<RCC>
    <qresource>
        <file alias="/fa/solid/folder-open.svg">../libs/font-awesome/svgs/solid/folder-open.svg</file>
        <file alias="/bi/floppy-fill.svg">../libs/bootstrap-icons/icons/floppy-fill.svg</file>
    </qresource>
</RCC>

The font-awesome and bootstrap-icons directories are not deployed with the applications as it would be a huge bloat (in the end we use maybe 1% of their icons). So those directories do not exist at runtime anymore.

Now my idea would be to add all images which are used by Slint, to the *.qrc file too. That's a bit cumbersome, but I think that can be automated with a little Python script - problem solved. Then Slint should load the images from Qt's resource system (path rewriting from "../libs/bootstrap-icons/icons/floppy-fill.svg" to "/bi/floppy-fill.svg" is easy).

Due to this complexity, and probably many different needs/opinions from other application developers, I came to the idea of having a callback for the resource loading. I think it would be a very flexible way, allowing to implement basically every imaginable concept. No matter how complex the final workflow is, Slint's API would be simple and clean, leaving the complexity (or simplicity) up to the application developer.

[tronical]

I don't quite like the idea of exposing the different types of resources to the app developer. My preference would be that idea with providing a base directory and the run-time library calling into the backend to return the Box<[u8]>. With the Qt backend that would use QFile and thus the resource system would automatically work. However, that implies probably shipping everything in resources.

Is there a particular reason for mixing the deployment methods (some compiled into the app as resources, some deployed by the side)?

[ubruhin]

My preference would be that idea with providing a base directory and the run-time library calling into the backend to return the Box<[u8]>. With the Qt backend that would use QFile and thus the resource system would automatically work. However, that implies probably shipping everything in resources.

Hmm but that means mixing the resource system with the backend selector, which sounds like an unnecessary complexity to me. I'd prefer to handle resources the same way for all backends to keep things independent of each other.

Is there a particular reason for mixing the deployment methods (some compiled into the app as resources, some deployed by the side)?

Generally this was just the easiest way to set up for us. With some effort (and possibly small drawbacks) we could bundle either all or nothing. But honestly I feel like an UI framework should not put any (opinionated) constraints about how we deploy our own files.

I don't quite like the idea of exposing the different types of resources to the app developer.

My preference would be that idea with providing a base directory [...]

I understand that, and generally it doesn't mean we cannot handle resource types differently. In the end, we could always do something like this:

std::vector<uint8_t> slint_callback_to_load_ressource_file(std::string filepath) {
  if (filepath.starts_with("fonts/")) {
    return load_file_from_share_directory(filepath);
  } else {
    // Rewrite paths for the Qt resource system.
    filepath.replace("font-awesome/svgs/", ":/fa/");
    filepath.replace("bootstrap-icons/", ":/bi/");
    return load_file_from_qrc(filepath);  // Uses QFile to read resource files
  }
}

So Slint can assume that at runtime there is the same directory structure as at compile time. It just shall not load the files directly, to allow us redirecting the calls to whatever system we use for resource loading.

EDIT: Btw, what I don't know is whether simply returning a bytearray is actually good enough or if some kind of stream readers would be required, e.g. for reading large font files.