Use size_t arithmetic for pixel offset calculations in reformat.c

[uwezkhan]

This change promotes several pixel offset multiplications in avifImageRGBToYUV and avifImageYUVAnyToRGBAnySlow to size_t before pointer arithmetic.

Currently, expressions such as:

(i * rgbPixelBytes)

are evaluated using 32-bit arithmetic before being promoted to size_t. While current AV1 frame dimension limits prevent overflow under normal conditions, performing these calculations directly in size_t ensures correctness and avoids reliance on implicit width constraints.

[wantehchang]

uwezkhan: Thank you for the pull request. We are in the middle of creating the libavif v1.4.0 release. I will review this pull request next week.

[wantehchang]

Please run clang-format on this file:

clang-format -i --style=file src/reformat.c

or apply the following patch:

diff --git a/src/reformat.c b/src/reformat.c
index a487a724..09dacf91 100644
--- a/src/reformat.c
+++ b/src/reformat.c
@@ -322,7 +322,8 @@ avifResult avifImageRGBToYUV(avifImage * image, const avifRGBImage * rgb)
                         if (alphaMode != AVIF_ALPHA_MULTIPLY_MODE_NO_OP) {
                             float a;
                             if (state.rgb.channelBytes > 1) {
-                                a = *((uint16_t *)(&rgb->pixels[offsetBytesA + ((size_t)i * rgbPixelBytes) + (j * rgbRowBytes)])) / rgbMaxChannelF;
+                                a = *((uint16_t *)(&rgb->pixels[offsetBytesA + ((size_t)i * rgbPixelBytes) + (j * rgbRowBytes)])) /
+                                    rgbMaxChannelF;
                             } else {
                                 a = rgb->pixels[offsetBytesA + ((size_t)i * rgbPixelBytes) + (j * rgbRowBytes)] / rgbMaxChannelF;
                             }
@@ -807,8 +808,10 @@ static avifResult avifImageYUVAnyToRGBAnySlow(const avifImage * image,
                         unormV[1][0] = *((const uint16_t *)&vPlane[(uvJ * vRowBytes) + ((size_t)uvI * yuvChannelBytes) + vAdjCol]);
                         unormU[0][1] = *((const uint16_t *)&uPlane[(uvJ * uRowBytes) + ((size_t)uvI * yuvChannelBytes) + uAdjRow]);
                         unormV[0][1] = *((const uint16_t *)&vPlane[(uvJ * vRowBytes) + ((size_t)uvI * yuvChannelBytes) + vAdjRow]);
-                        unormU[1][1] = *((const uint16_t *)&uPlane[(uvJ * uRowBytes) + ((size_t)uvI * yuvChannelBytes) + uAdjCol + uAdjRow]);
-                        unormV[1][1] = *((const uint16_t *)&vPlane[(uvJ * vRowBytes) + ((size_t)uvI * yuvChannelBytes) + vAdjCol + vAdjRow]);
+                        unormU[1][1] =
+                            *((const uint16_t *)&uPlane[(uvJ * uRowBytes) + ((size_t)uvI * yuvChannelBytes) + uAdjCol + uAdjRow]);
+                        unormV[1][1] =
+                            *((const uint16_t *)&vPlane[(uvJ * vRowBytes) + ((size_t)uvI * yuvChannelBytes) + vAdjCol + vAdjRow]);
 
                         // clamp incoming data to protect against bad LUT lookups
                         for (int bJ = 0; bJ < 2; ++bJ) {