Support for files produced by AvaSoft 8.4

[Bisaloo]

This issue aims at summarizing the history and current situation about support for files produced by AvaSoft 8.4

[Bisaloo]

• Versions of lightr up to 1.7.0 supported binary files produced by all versions of AvaSoft 8 (IRR8, RFL8, RAW8), without restrictions
• In May 2022, I got my hands (how?) on IRR8 files produced by AvaSoft 8.4 that were parsed by lightr without any warnings, but produced unexpected results. As a reaction, support for AvaSoft 8 was restricted to versions where I could confirm the match between lightr-parsed and officially-parsed files: cd180d3 (i.e., only AvaSoft 8.2 at that time).
• In February 2023, I got an email from another user that requested restoring parsing. After some back and forth, they sent me the officially-parsed Raw8 files for comparison in June 2023.
  Interestingly, these files were also produced by AvaSoft 8.4.
  There were producing "correct" outputs, with the caveat that the official parser placed the "scope" column as "processed", which makes sense because these were scope measurements, as confirmed by the fact that the internal measmode var is set to 0.
• In February 2025, user request to reactivate support for RFL8 files from AvaSoft 8.4
• In August 2025, user request to reactivate support for RFL8 files from AvaSoft 8.4
• In October 2025, user request to reactivate support for RFL8 files from AvaSoft 8.4

[Bisaloo]

While digging more in the IRR8 AvaSoft 8.4 files that were producing unexpected output:

• the scope, dark and white values match
• the "processed" value doesn't match. I can re-create a nearly identical spectral shape (with a seemingly random scaling factor) by doing (scope-dark)/white.

I'm still not clear:

1. Why this formula is used
2. How to determine the scaling factor

[Bisaloo]

Here is the header of the officially-parsed .Raw8 Avasoft 8.4 file:

Wave   ;Sample   ;Dark     ;Reference;Scope
[nm]   ;[counts] ;[counts] ;[counts]

300.014;  268.225;  248.416; 3717.762

They're listing 5 columns in the table header but then only return 4 columns 😭

[Bisaloo]

Update on this: I got more requests for support & test files from different users.

I can confirm a perfect match between was we return and what the official parser returns (modulo a difference in normalization choices for very low wl but data is unusable in all cases).

It seems the mismatch I saw before only affects .IRR8 and not reflectance files. Avantes apparently has a different way of normalizing irradiance files that I need to figure out.