Feature request: produce high bit depth buffers

[mstoeckl]

Some wallpapers are better viewed at higher bit depths than the usual 8bpc -- for example, cameras often capture 10bpc data or higher, artwork and rendered images have >=16bpc working spaces, and the gradients in SVG images need high bit depths to avoid banding (or they must use dithering, which has other downsides). Compositors supporting wlr-layer-shell like Sway or Kwin already support 10-bit depth on supported hardware, and may use higher bit depths in the future. swaybg will need to load, process, and output high bit depth image data for the entire pipeline to work.

swaybg's current default image loading library is GdkPixbuf, which only produces 8-bpc unpremultiplied images and is unlikely to ever support anything else. To render backgrounds, it uses Cairo, which can composite images using 8-bpc, 10-bpc, and (pending 1.18.0 release, 32-bit-float-per-channel).

High bit depth inputs:

• PNG (u16)
• AVIF (u10)
• EXR (fp16, fp32, u32)
• SVG (can be rendered to anything, in theory; librsvg + Cairo can do f32)
• TIFF (everything)

High bit depth shared memory formats that can be output:

• u10 (XBGR2101010, ...)
• f16 (XBGR16161616F, ...)
• u16 (XBGR16161616, ...)

Long term, changes which make color management easier would also be nice.

Approaches:

• Directly using PNG/EXR/SVG libraries with Pixman or Cairo: This would require specific code for each new format, but support for these can be added incrementally. Existing formats can use GdkPixbuf. Switching to composite images directly with Pixman instead of Cairo may increase the range of supported formats and avoid having to write some format conversion logic for e.g. u16->u10. Note that Cairo's higher bit depth formats (RGB30, RGB96F, RGBA128F) either have no Wayland equivalent or theirs, like XRGB2101010, is rarely available. Demo branch at mstoeckl/swaybg/pixman-3.
• GEGL: Can load most formats, and can do rendering. Rather slow to start (gegl_init takes a full 0.1-0.2 seconds to run), and less efficient than Cairo/Pixman (sRGB<->linear conversions are expensive), but can multithread. Does have support for color spaces. Only supports byte-aligned formats, i.e., u8.u16.f16, but not u10 (although in theory any format can be be used with enough custom loading code). Demo branch at mstoeckl/swaybg/gegl.
• VIPS: Can load most formats and do rendering. Slow to start cold due to linking with many libraries. Supports u16 as output format, but not f16 or u10. May be efficient at loading large images -- although technically users should provide properly sized images to begin with. Demo branch at mstoeckl/swaybg/vips.
• glycin + Cairo: glycin is a successor to gdk-pixbuf, has sandboxed image loaders, supports high bit depths (can produce a variety of pixel formats, and since 2.0 can convert between them). (gdk-pixbuf has switched to this as a backend.) Now exposes CICP in its C API, but not ICC. Demo branch at mstoeckl/swaybg/glycin.

Other libraries with C APIs that are used for either image loading or rendering:

• freeimage: handles image loading and rendering, provided under GPLv2/GPLv3; does loading/writing and format conversions for u16,u32,f32, and can rescale these at precision, but FreeImage_Composite only does u8 precision
• imlib2: context-dependent API; appears to only support 8-bit depth, no color management
• skia: used statically for rendering by Chromium, but not otherwise in distributions; supports u10/f16/u16
• imagemagick: with MagickCore API can load and render, supports u16,f16 and color managment, custom license
• libplacebo: GPU oriented, handles scaling/compositing for mpv; has proper color management; unclear if CPU-only supported