RGBGFX segfault

[Rangi42]

Discord report: https://discord.com/channels/303217943234215948/870005582042628196/1336507100620132382

The gist of it seems to be that embedded palettes are not read properly, and an image with an embedded palette that uses color #0 on any pixel causes the code that displays a palette to segfault. Here's the output for rgbgfx -o segfault.2bpp -c embedded segfault.png:

error: Failed to fit tile colors [$1527, $15cc, $1ab4] in specified palettes
note: The following palette was specified:
Segmentation fault (core dumped)

If I modify the image to not use color #0 from the embedded palette, I get a similar failure but with no segfault, so the code successfully prints... no palette:

error: Failed to fit tile colors [$1527, $15cc, $1ab4] in specified palettes
note: The following palette was specified:
        []
Conversion aborted after 1 error

[Rangi42]

Note that this bug does not occur with the Rust rewrite:

$ target/debug/rgbgfx -c embedded -o segfault.2bpp segfault.png 
error: Some tiles cannot be displayed with the specified palettes
  │ 
  │ Note: The following palettes were specified:
  │         [transparent, $1ab4, $15cc]
  │       No palette contains colors [$1527, $15cc, $1ab4]
──╯

[Rangi42]

I'm not sure if the "Failed to fit tile colors"/"Some tiles cannot be displayed" error is appropriate though. The embedded palette specifies four colors, with transparency as a fifth:

% pngcheck -p segfault.png
File: segfault.png (279 bytes)
  PLTE chunk: 4 palette entries
    0:  (108,224, 64) = (0x6c,0xe0,0x40)
    1:  (160,168, 48) = (0xa0,0xa8,0x30)
    2:  ( 96,112, 40) = (0x60,0x70,0x28)
    3:  ( 56, 72, 40) = (0x38,0x48,0x28)
  tRNS chunk: 4 transparency entries
    0:    0 = 0x00
    1:  255 = 0xff
    2:  255 = 0xff
    3:  255 = 0xff
OK: segfault.png (96x8, 8-bit palette+trns, non-interlaced, 63.7%).

(Note that those four RGB888 palette entries are equivalent to GBC [$238d, $1ab4, $15cc, $1527].)

The actual image clearly uses four colors: three from the palette (indexes 1, 2, and 3) plus transparent. So the question is, should "transparent" be counted as the unused color at index 0, or should it fail because -c embedded says "use this exact palette" and transparent is not the same color as (108,224, 64)?

This also makes me question the Rust output: why is it claiming that the specified palette is [transparent, $1ab4, $15cc] instead of [$238d, $1ab4, $15cc, $1527] (the embedded PLTE) or [transparent, $1ab4, $15cc, $1527] (the four actually-used colors)?

[Rangi42]

Here's the abbreviated make develop ASan output:

$ ./rgbgfx -c embedded -o segfault.2bpp segfault.png
=================================================================
==22676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000298 at pc 0x00010d93b93a bp 0x7ffee2333790 sp 0x7ffee2333788
READ of size 2 at 0x602000000298 thread T0
    #0 0x10d93b939 in _ std::find<_>(_) algorithm:929
    #1 0x10d93b6e5 in makePalsAsSpecified(_)::$_0::operator()(_) const::'lambda'(_)::operator()(_) const process.cpp:633
    #2 0x10d93b481 in bool std::all_of<_>(_) algorithm:855
    #3 0x10d93b302 in makePalsAsSpecified(_)::$_0::operator()(_) const process.cpp:632
    #4 0x10d935bd1 in _ std::find_if<_>(_) algorithm:943
    #5 0x10d92c4cd in makePalsAsSpecified(_) process.cpp:630
    #6 0x10d92dd0d in process() process.cpp:1233
    #7 0x10d8a59e8 in main main.cpp:881
    #8 0x7fff208e2f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

0x602000000298 is located 0 bytes to the right of 8-byte region [0x602000000290,0x602000000298)
allocated by thread T0 here:
    ...
    #8 0x10d92c2a6 in makePalsAsSpecified(_) process.cpp:605
    #9 0x10d92dd0d in process() process.cpp:1233
    #10 0x10d8a59e8 in main main.cpp:881
    #11 0x7fff208e2f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)