The conversion from image data to raw argb values is incorrect. color.RGBA() returns 4 uint32 values that must be divided by 256 to scale them back to the range 0-255 rather than simply truncating via a byte() cast.

If I print just the r values you can see how the truncation differs from the division result.

2025/03/17 09:23:33 r=771 r&255=3 r/256=3
2025/03/17 09:23:33 r=1285 r&255=5 r/256=5
2025/03/17 09:23:33 r=1028 r&255=4 r/256=4
2025/03/17 09:23:33 r=834 r&255=66 r/256=3
2025/03/17 09:23:33 r=10 r&255=10 r/256=0
2025/03/17 09:23:33 r=362 r&255=106 r/256=1
2025/03/17 09:23:33 r=7830 r&255=150 r/256=30

sometimes its the same (weird coincidence) but some values are off.