Add lumen-gi-enabled / lumen-reflections-enabled camera capture settings (captures match the viewport by default)

[andrewjong]

About

TL;DR adds Lumen support to AirSim camera images.

Camera images captured through USceneCaptureComponent2D don't render with the global illumination / reflection method the viewport uses: a capture view seeds its method from the project CVars but never blends level PostProcessVolumes, and on hardware-ray-tracing projects the capture's rays have no acceleration-structure data unless the component opts in. The practical result is that a Lumen-lit level comes out flat and washed out in the sim's camera images — all indirect light missing — even though the game/editor viewport looks correct (see screenshots below).

Since UE 5.5 the engine natively supports Lumen in scene captures: when a capture's post-process settings resolve to Lumen, the renderer maintains a dedicated Lumen scene for the capture's persistent view state (which UnrealCamera already enables via bAlwaysPersistRenderingState).

This PR adds two per-capture tri-state settings, documented in docs/sensors/camera_capture_settings.md:

• Key absent (the default): match the viewport. Scene (RGB) captures mirror the project's r.DynamicGlobalIlluminationMethod / r.ReflectionMethod, so Lumen projects get Lumen camera images out of the box. Every other image type (depth, segmentation, ...) resolves to None — their output comes from a replacement material, and previously they silently inherited the project CVar, paying Lumen cost for nothing.
• lumen-gi-enabled / lumen-reflections-enabled = true: force Lumen for that capture regardless of the project default.
• false: force the method to None. This is an active override — an un-overridden capture inherits the project CVar, so omitting the override is not an off switch.

The resolved methods are pinned as explicit post-process overrides on each capture, and bUseRayTracingIfEnabled is set whenever either method resolves to Lumen so hardware-ray-traced Lumen works (software Lumen works without it). The RGB wire format is unchanged (SCS_FinalColorLDR, 8-bit — in UE 5.5+ it runs the full viewport film tone curve). On engines older than 5.5 the overrides are ignored for scene captures, so this is a no-op there.

Cost note: each Lumen-enabled capture maintains its own Lumen scene (GPU time + VRAM) — numbers below; set the knobs to false per capture to opt out where frame rate matters more than fidelity.

How Has This Been Tested?

UE 5.7.4, Vulkan SM6, Linux (Epic unreal-engine:dev-5.7.4 container), photorealistic office scene (marketplace "QA Office" level, Lumen GI + reflections + HWRT in project settings), non-physics robot with two 640×480 RGB+depth-planar cameras at a 0.1 s capture interval:

• Fixed-pose A/B across all three states (same build, only the jsonc differs): keys absent → captures match the viewport's lighting (Lumen); false → matches the pre-PR captures (verified visually and by luminance statistics); true → forces Lumen. The on/off pair was verified at four camera poses, the full tri-state matrix at one.
• Quantitative: a ceiling lit only by GI bounce goes from 59/255 to 106/255 mean luminance (global mean is unchanged because auto-exposure renormalizes — the effect is in the light distribution and reflections).
• Cost measured: two 640×480 Lumen RGB captures: 4.2 → 3.8 Hz capture rate (~25 ms/frame cycle) and +0.5 GiB VRAM.
• Depth-planar output verified unchanged alongside the RGB runs.

Screenshots and videos:

Lobby — ceiling receives no direct light, only GI bounce; the marble floor gains reflections:

Lounge — ceiling, brass stanchions, chrome table legs:

Corridor — the trash can turns from flat matte to brushed metal with environment reflections:

Game-viewport reference at the same pose as the corridor pair — the Lumen-on capture matches its lighting (the yellow on-screen text is an unrelated engine debug message, viewport-only):

🤖 Generated with Claude Code

[LucasJSch]

Nice feature! Thanks for your contributions @andrewjong !

A few comments:

• Can you try these changes on UE5.2, and let me know if it truly is a no-op there? If not, it should be guarded with #if UE_IS_5_7 or something of the sort.
• “Match viewport” does not include Post Process Volumes. A viewport can have those project defaults overridden by a bounded or unbound Post Process Volume. The capture will still use the project CVar, so these cases will not match:
  • Project default None, volume selects Lumen.
  • Project default Lumen, volume disables it.
  • Different volumes select different methods by camera location.
• Add schema-checked variables for the new parameters. I would add:

"lumen-gi-enabled": { "type": "boolean" },
"lumen-reflections-enabled": { "type": "boolean" }

• Can you benchmark the cost of non-static cameras also? According to the UE documentation this can affect the cost of using Lumen.

Regarding changing default cameras behavior: @jonyMarino this is your call.

[andrewjong]

Will do! I'm currently focusing on ICRA so probably this will come late September.

[jonyMarino]

Glad to hear. Are you going to present a work using Project AirSim?

[andrewjong]

Yes possibly! Would you have a recommended bibtex citation?