RFC: Linear Color Space Rendering Support for HDR Workflows

[akre54]

Forgive the AI slop but it worded it better than I could. Basically I want to be able to render out layers and passes individually to EXR files, and do color correction later in external programs like DaVinci Resolve, Nuke, etc.

Taking a page from Blender and other renderers (and after a conversation this morning with @donmccurdy), we can get rendering appearance benefits by keeping math in linear color space and then applying sRGB correction when rendering to the screen.

Background

deck.gl currently renders in gamma-encoded sRGB color space. This is evident in several places throughout the codebase:

1. Opacity gamma correction (modules/core/src/shaderlib/misc/layer-uniforms.ts):

   opacity: Math.pow(props.opacity!, 1 / 2.2)

   This applies gamma correction to opacity values to make them appear perceptually linear on screen.

2. Texture handling: Textures are loaded and processed assuming sRGB encoding.

3. Color blending: All color operations happen in gamma-encoded space.

While this works well for standard web visualization, it creates challenges for professional workflows that require physically accurate rendering.

The Problem

When deck.gl outputs are composited with other 3D content (film/VFX pipelines, architectural visualization, scientific visualization), the gamma-encoded output leads to:

• Incorrect color blending when layered with linear-space renders from other tools
• Clipped highlights since 8-bit sRGB can't represent values above 1.0
• Color banding in subtle gradients due to limited bit depth
• Incompatibility with HDR displays and formats (EXR, HDR10, etc.)

For physically-based rendering workflows (like those using ScenegraphLayer with PBR materials), rendering in sRGB space undermines the physical accuracy that PBR is designed to provide.

Use Cases

1. Film/VFX Compositing: Exporting deck.gl renders to EXR for compositing with live action or other CG elements
2. Architectural Visualization: Combining geospatial deck.gl layers with Blender/Unreal renders
3. Scientific Visualization: Preserving full dynamic range for analysis (astronomy, medical imaging)
4. HDR Display Output: Supporting HDR monitors and video formats

Proposed Solution

Add opt-in linear color space rendering support:

1. New layer prop: _linearColorSpace: boolean (underscore prefix for experimental API)

   • When true, skip gamma correction on opacity
   • Signal to shaders to expect/output linear values

2. Color space conversion utilities: GLSL/WGSL functions for sRGBToLinear() and linearToSRGB() using the official IEC 61966-2-1 transfer function

3. Float framebuffer rendering: Leverage existing rgba16float/rgba32float support in luma.gl for HDR output

4. HDR export utilities (optional): Helper classes for exporting to HDR formats

Questions for Discussion

1. API Design: Should this be a per-layer prop (_linearColorSpace) or a global deck option? Per-layer offers flexibility but may cause inconsistencies when layers mix color spaces.

2. Default Behavior: Should we consider making linear color space the default in a future major version (v10)? The current gamma-corrected opacity was likely a pragmatic choice that may not age well.

3. WebGPU Considerations: WebGPU has better support for sRGB texture formats and automatic gamma correction. Should we design the API with WebGPU's capabilities in mind?

4. Scope: Should the initial implementation focus narrowly on "opt-in linear output" or attempt to handle input textures (converting sRGB textures to linear on load)?

5. Interaction with existing PBR: The ScenegraphLayer already uses PBR through luma.gl. How should linear color space mode interact with existing PBR materials?

Prior Art

• Three.js: renderer.outputColorSpace = THREE.LinearSRGBColorSpace
• Babylon.js: scene.imageProcessingConfiguration.toneMappingEnabled
• MapLibre/Mapbox: Generally sRGB-only, similar to current deck.gl

I'd love to hear thoughts from the maintainers and community on the best approach here.

[akre54]

I'm trying to focus on general rendering improvements but I'm probably out of my depth here. I'm using these as a starting point:

• https://www.w3.org/Graphics/Color/Workshop/slides/talk/russell
• https://codesandbox.io/p/sandbox/building-live-envmaps-lwo219
• https://www.npmjs.com/package/postprocessing
• Use the P3 color space with the ‘display-p3’ option to gl.drawingBufferColorSpace.

An easy win might be to use HDRI image textures for lighting. @tsherif says this is the most straightforward way to “plus” a render in Deck. It will give more naturalistic light and make the colors more interesting. Deck supports HDRIs in ScenegraphLayer with _imageBasedLightingEnvironment and the pbr _lighting option

[donmccurdy]

For one point of reference, three.js added color management and a linear workflow incrementally, both enabled by default with release r152. The roadmap issue and migration guide give some idea of the changes involved, and results before/after:

• Roadmap for a color-managed workflow in three.js mrdoob/three.js#23614
• https://discourse.threejs.org/t/updates-to-color-management-in-three-js-r152/50791

The color management guide in Blender is a good resource as well. Essentially, every color going into deck.gl would need a defined color space (this could be as simple as documenting that all inputs are assumed to be "sRGB", which they effectively are today), converted to some working color space (probably "Linear-sRGB") for use in lit material shaders, then tone mapped and converted from the working color space to an output color space (typically "sRGB") before writing to the canvas drawing buffer. Ideally the "working color space" and "output color space" are configurable, where setting both to "sRGB" should be identical to what deck.gl does today, and no conversions are needed. Also see:

• https://github.com/mrdoob/three.js/blob/dev/src/math/ColorManagement.js
• https://github.com/mrdoob/three.js/blob/dev/examples/jsm/math/ColorSpaces.js

Very workable on WebGL 2; no particular need for WebGPU-specific features. It isn't necessary to render to an intermediate float16 or float32 framebuffer, unless you want to do the final tonemapping and working-to-output color space conversion in a separate pass.

That said, most of the linear workflow's benefits are oriented toward illuminated scenes and shading models. Many of deck.gl's common use cases don't involve a shading model, and while there might be some benefit for these unlit scenes too... I think those effects would be difficult to notice. So that may be a consideration on the cost/benefit side of things.

Aside - opacity is only related as historical accident of #8972. Normally a linear workflow does not need to change how opacity works at all, opacity does not need to be gamma corrected or converted between color spaces.