Add a Lua API for Images

[appgurueu]

Problem

We have dynamic media, and helpers like core.encode_png to write images. But where should the image to be encoded come from? Often we would like to take some image, do something with it (e.g. extract some information or modify it), then write a new image.

Epidermis for example loads skin images, lets you edit them in-game, and uploads new images via a mixture of dynamic media and texture modifiers (small changes use texture modifiers, large changes send a new image). Ghosts also reads skins (and model files), but for a different purpose: It approximates player appearances using particles to create a ghost-like visual.

Solutions

I propose an Image Lua "class" backed by userdata, which under the hood would probably wrap Irrlicht images. It should be possible to:

• Load an image from one of the supported file formats (PNG / JPG / TGA);
• Create an image of given dimensions, filled with a single color;
• Get/set single pixels (ARGB8 colors);
• Get/set pixels in bulk, as a table of ARGB8 numbers, for fast bulk operations;
• Write an image in PNG or JPG format (TGA is less important)

We should also think about various 2d array operations (e.g. slicing) and drawing, but those can come later.

Alternatives

Bring your own loaders and utilities. This is what Epidermis and Ghosts do. They use a pure Lua PNG reader in modlib. However, this has some limitations:

• It is not competitive performance-wise.
• The more complex JPG file format can not be supported easily.
• While I have tested it on PNGSuite, it has not seen the same amount of battle-testing as libpng.

To me, image facilities fall squarely in the game engine scope; while they can be provided by mods in some capacity, they should not have to be.

Additional context

• We will likely want to base this upon Irrlicht's images, which should be refactored first: Refactor images: Base them on 2d arrays #16364
• Regarding a general canvas abstraction: 2D canvas API feature #16643 (comment)
• Images can be large; similar to VoxelManips, we should think about a :close() method to drop memory early, as Lua's GC will be unaware of the real size.
  • Or we could think about turning images into a single allocation, which Lua would then be aware of.
    • But then we need to separate images (owning) and image views (non-owning), though maybe that can be simplified with templating somehow...
• core.encode_png should be changed to accept an image object as well, or be deprecated / discouraged in favor of the equivalent image API.
• Latest example of a use case where someone seems to want to extract some information from an image: Add functions for color detection in textures #16793
• Additionally, for convenience, it should be possible to get media paths and/or contents on the server (details esp. w.r.t. dynamic media t.b.d.).
• This goes for other forms of media too, namely models and sounds, but one issue at a time.
  • I have a WIP branch with skinned mesh writers implemented in C++ for the gltf and b3d file formats.
  • I plan to write Lua-side abstractions for skinned meshes.

[sfan5]

this has some overlap with #14477 and similar requests. the very first roadblock is that the Luanti server is not linked to libjpg or libpng at all.

[farooqkz]

I think this would come very handy.

Additionally, Do you folks think it will be useful to do a screenshot(without chat and HUD) and then send it to the server? The use case I have in mind, is that a player could see what the other player sees. It's both useful for moderation purposes and for creating new gameplays.

[sfan5]

Do you folks think it will be useful to do a screenshot(without chat and HUD) and then send it to the server?

no: #16398