Rendering interface command list

[lan-shichao]

Can we consider converting all rendering into command lists, and then rendering the command lists in the rendering backend, similar to imdrawlist in "dear imgui"?

[wh1t3lord]

@lan-shichao formally yes, and it might be useful (???) at some point, but idk how it can be used by user if RmlUi has tendency to keep things implicit like user operates with HTML/CSS not with render interface at all. I guess it won't give much for us and for users, otherwise provide use cases where user can define custom render route with pre-defined render commands (does HTML/CSS provide a such thing?).

Otherwise it formulates a problem that RmlUi's render interface should give user the ability to write custom rendering "effects" with defined rendering commands.

So from that perspective there would be a lot of work, users need to give feedback and reasons why they want to have rendering commands and having ability to provide own rendering routes.

I would think about batch rendering of current backend implementations instead of thinking about these commands, but again if community will request a such thing and that thing will be in high demand then okay, maybe we will implement some day, but formal logic suggests that RmlUi wants to give HTML/CSS functionality at first place and manupilating rendering is kinda ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯.

Ofc we can make a reference to another drawing backend as Skia that used a lot by webkits, but Skia is not UI library it is just a draw library that can be used to make UI library, by that logic they have abstract rendering commands and user can define own "rendering logic" but it has different ideology compared to RmlUi.

Also keep in mind that we need to give users freedom of editing our backends and users want to transform backend as they want in their engines/3d/2d visualization programs (if they don't want to use feature like integrate backend out-of-box). If we put rendering commands it gives complex logic and it becomes a hard task for user to edit backend like for now it is simple and trivial (at some degree) but then backend becomes a large thing and a lot of implicit logic. ImGui backends just go as drag-n-drop but users don't edit them (most of the time), however for a long time users edited RmlUi's backends that we provided and it was okay, so I have some doubts about that feature.

If my reasoning was valid for you then okay if not then elaborate please.

Best regards

[lan-shichao]

First, we will adjust the rendering process into a command list, treating drawing commands, scissor commands, adjustment commands, etc., as a command stream.
Next, we will use the existing RmlUI graphics backends (such as OpenGL, DirectX, and Vulkan renderers) to execute these commands.
This successfully introduces an intermediate process between rendering calls and rendering implementation.

With this in place, we can integrate 2D rendering engines like Skia or Cairo as renderers, functioning similarly to the existing RmlUI graphics backends.
This brings additional benefits: we will be able to issue commands on one terminal and render them on another, similar to Windows RDP.
Or even server-side rendering?

Since we can connect any rendering backend, we can start using more performant or better-compatible rendering methods like D2D or GDI at the native layer.

Furthermore, we can attempt to decouple the CSS mechanism from the HTML mechanism, making the HTML part an independent and highly flexible DOM tree executor. This will result in a framework structured as follows:
"Styling Frontend" (CSS/RCSS or other styling description interfaces) → DOM Executor (a purely headless UI description structure) → Any Renderer.

I aim to make RmlUI a cross-platform native UI library similar to Flutter but with greater flexibility.
In my vision, I would be able to write JSX code to drive the RmlUI core and set high-level abstract styling properties to change the appearance of controls (instead of writing extensive CSS for UI styling), while achieving native performance and compatibility.

Thanks to the flexibility of the "Styling Frontend," RmlUI can still maintain its pursuit of web standards. The implementation of CSS will become one specific realization within the "Styling Frontend" of the "future RmlUI."

This may sound far-fetched, and there are certainly challenges, but I will try to address them. Even without community support, I will still attempt to implement it and am open to suggestions and ideas from the community and you.

This will be a long-term effort, but it is essential to gather discussions and opinions before starting.

[mikke89]

I made some experiments at implementing a command list some while ago. See the discussion here: #440

I didn't pursue it any further. I figured, it's not really worth the trouble. You have to remember, a command list, or an interface, it's just an API regardless. It really just comes down to preference. You could always make an adapter if you have a strong preference here, i.e. batch up all the render interface calls into your own commands list. And there is nothing about the render interface that limits the implementation of a Skia or Cairo backend.

Maybe if we started from scratch I would have made a different choice, but I'd argue a command list doesn't provide much added value. And we can't have both, so it's better to stick with what we have.