Fix 32-bit overflow in LZW size calculation (MAX_CODE_LEN * lzwPos)#108
Fix 32-bit overflow in LZW size calculation (MAX_CODE_LEN * lzwPos)#108uwezkhan wants to merge 3 commits into
Conversation
|
Hey @uwezkhan, Missing test case (required): Bug fix PRs must include a test in AI tooling attribution: If any AI models or tooling were used to produce this PR, please disclose that per our contributing guidelines (CONTRIBUTING.md:, item 5). |
At 19000x19000 the LZW code count landed just under UINT32_MAX/12, so the 32-bit product never wrapped and the test passed on unfixed main. Bump to 20000x20000 so lzwPos stays above the threshold and the overflow reproduces.
|
Added the reproducer as Proof of failure, same test run against both trees: Before (unfixed main): After (this branch): The test encodes 20000x20000 pseudo-random pixels so the LZW code count stays above UINT32_MAX/12, which is where the 32-bit On the tooling question: yeah, I lean on an LLM a bit for research and sanity-checking, nothing heavy. |
2 participants
This patch fixes a 32-bit arithmetic overflow in the LZW
byte list size calculations in cgif_raw.c.
The expression:
was evaluated using 32-bit arithmetic because
MAX_CODE_LEN is defined as an int and lzwPos is uint32_t.
Under C's usual arithmetic conversions, the multiplication
is performed as uint32_t before being widened to 64-bit
for the subsequent division.
If lzwPos is sufficiently large, the intermediate
32-bit multiplication can wrap around, resulting in an
incorrect (truncated) buffer size being passed to malloc().
The fix casts MAX_CODE_LEN to uint64_t:
This ensures the multiplication is performed in 64-bit
arithmetic and prevents intermediate overflow.
No API changes.
No behavioral changes for valid inputs.
Negligible runtime impact.