Skip to content

Calibrated pictures lose their provenance header, and leak absolute paths #241

Description

@adulbrich

Found while comparing a CR2 run against a JPEG run for the WebAssembly port. Affects both pipelines identically, and has nothing to do with the port: it is how the pipeline has always behaved.

The finished picture keeps less provenance the more calibration you apply

Two runs from the same day, both through the whole pipeline. The only relevant difference is that one had calibration files selected and one did not. Via exiftool:

CR2 run, no .cal files JPEG run, with CF_f5d6.cal
Camera Canon EOS 5D Mark III version dcraw v9.26 absent
Capture date 2020:11:23 12:31:37, the actual capture 2026:07:29 11:26:02, the processing time
PRIMARIES present absent
EXPOSURE 1.0000e+00 absent
hdrgen provenance names all 10 merged frames, plus Removed lens flare absent
Crop and resize pcompos -x 3740 -y 3740 ... -1026 -69, pfilt -1 -x 1000 -y 1000 absent
What survives all of the above one pcomb command line

So a reader of the calibrated picture cannot tell which camera took it, which frames were merged, when it was actually shot, or that lens flare was removed. The uncalibrated picture records all of it. That is backwards for a tool whose outputs go into papers.

Cause

pcomb -h. In pcomb.c:118:

case 'h':
    echoheader = !echoheader;

echoheader defaults to 1, so -h turns it off and the input's header is never copied through (pcomb.c:334, :351). It does not suppress pcomb's own command line, which is why that one line is all that remains.

The part that looks accidental: only the last of the four pcomb stages passes it.

Stage -h?
projection_adjustment.rs no
vignetting_effect_correction.rs no
neutral_density.rs no
photometric_adjustment.rs:20 yes

The first three accumulate header lines and the fourth discards them all. If a clean header were the goal, -h would be on all four; if provenance were the goal, on none. And because the four stages only run when calibration files are supplied, a run without them keeps everything by default rather than by design.

Confirmed across all eight JPEG runs and both CR2 runs on the machine: pcomb present implies EXPOSURE absent, without exception.

Nothing numerical is at risk

Worth stating so this is not treated as more urgent than it is.

  • EXPOSURE is always 1. nullify_exposure_value runs ra_xyze -r -o, and -o sets origexp = 1.0 (ra_xyze.c:105). Forcing exposure to 1 is that stage's entire purpose. Every reader defaults a missing EXPOSURE to 1 anyway, including our own viewer (image-viewer/view/page.tsx:289).
  • PRIMARIES is always Radiance's default, because ra_xyze -r writes exactly those values. Dropping it changes no downstream interpretation.

Both of those hold by luck of this particular pipeline rather than by design, which is its own small argument for not relying on them.

A wrinkle: more provenance means more path leakage

Restoring the header is not purely additive, because pcomb's own command lines embed absolute paths. The current calibrated output already contains:

pcomb -h -f "/Users/<user>/Library/CloudStorage/GoogleDrive-<user>@oregonstate.edu/Shared drives/radiantlab HDRICalibrationTool/examples/inputs/calibration_files/CF_f5d6.cal" /work/neutral_density.hdr

That is a university email address baked into every calibrated picture, today, with -h on. Dropping -h would add three more lines of the same shape. Anyone publishing these files as supplementary material is publishing that.

So this issue is really two things, and the second may matter more than the first.

Options

  1. Drop -h. One-line change, full chain retained. Also triples the embedded absolute paths. Changes output headers for every calibrated run.
  2. Keep -h, re-add what matters via getinfo -a. The pipeline already appends VIEW= and COMPUTED_VERTICAL_ILLUMINANCE= this way (header_editing.rs), so the mechanism exists. Write the fields deliberately: camera, capture date, frame count, and calibration file basenames rather than full paths. More work, and the result is better than either current behaviour.
  3. Do nothing, document it. Defensible if nobody relies on picture headers for provenance. Worth asking before spending effort.

I would go with 2. It fixes the asymmetry and the path leakage together, and it puts what goes into the header under our control instead of leaving it to whichever tool happened to run last.

Whichever way it goes, -h should be consistent across the four stages rather than on one of them.

Notes

  • Both pipelines are affected identically. stages.ts:photometricArgs matches photometric_adjustment.rs byte for byte, deliberately, so this is not something the WebAssembly port introduced or can fix on its own.
  • stages.ts previously described -h incorrectly, as suppressing pcomb's own header line. Corrected in 0a76bf3, with no behaviour change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendbugsomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions