Integer Overflow and Out-Of-Bounds Heap Write in PSDDecoder

[YLChen-007]

Summary

An integer overflow vulnerability exists in the PSDDecoder component of the Filament engine's image processing library. When parsing a maliciously crafted .psd file with extremely large dimensions, the buffer allocation calculation overflows, leading to an undersized core allocation. A subsequent loop iterating over these massive dimensions triggers an unconstrained out-of-bounds heap write. This allows writing attacker-controlled file stream data out of bounds, which can lead to Denial of Service (DoS) or complete Remote Code Execution (RCE).

Details

The vulnerability originates in PSDDecoder::decode() within libs/imageio/src/ImageDecoder.cpp. It reads the width and height dimensions from the PSD header using ntohl() without validating their bounds:

        uint32_t width = ntohl(h.width);
        uint32_t height = ntohl(h.height);
// ...
        LinearImage image(width, height, 3);

Inside the LinearImage::SharedReference constructor (LinearImage.cpp), the heap buffer size is calculated by multiplying these dimensions with the channel count using 32-bit arithmetic:

    SharedReference(uint32_t width, uint32_t height, uint32_t channels) {
        const uint32_t nfloats = width * height * channels;
        float* floats = new float[nfloats];
// ...

By providing a crafted PSD with carefully balanced large dimensions (e.g., width=35031 and height=40866), the multiplication W * H * 3 wraps around the 32-bit integer limit ($2^{32}$) resulting in a tiny positive remainder (e.g., 66 floats). The undersized allocation succeeds.

Following the allocation, PSDDecoder parses the entire stream, calling LinearImage::getPixelRef for every pixel across all channels, writing the fully attacker-controlled file bytes out-of-bounds:

    for (size_t i = 0; i < 3; i++) {
        for (uint32_t y = 0; y < height; y++) {
            for (uint32_t x = 0; x < width; x++) { ...

Because the loop covers millions of executions, the out-of-bounds write sequentially corrupts heap blocks. This can be weaponized with precise algebraic calculations to target exactly the virtual table pointer of an adjacent C++ object dynamically allocated after the image buffer.

Impact

• Denial of Service (DoS): Widespread heap corruption easily causes application crashes whenever an affected application attempts to decode or process an attacker-supplied PSD image.
• Remote Code Execution (RCE): Control over the raw 32-bit floats written out-of-bounds allows complete memory forgery. Partial overwritten VTable pointers can lead to hijacking the execution flow (RIP) and jumping to user-sprayed execution blocks.
• Affected Surface: Any utility or host application (e.g., cmgen, mipgen) relying on libs/imageio for texture ingestion is vulnerable when loading untrusted PSDs.

Fix

A fix has been submitted via Pull Request. The mitigations involve:

1. Integer Overflow Prevention: Re-implement dimension multiplication inside LinearImage utilizing 64-bit integer (uint64_t) arithmetic and triggering std::bad_alloc if bounds exceed the safe threshold limit.
2. Dimension Limit Boundaries: Establish concrete maximum constraints on parsed variables inside PSDDecoder::decode() to prevent unbounded values (limiting both to < 300,000, matching adobe PSB spec).

CWE

• CWE-190: Integer Overflow or Wraparound
• CWE-787: Out-of-bounds Write

[YLChen-007]

Fix submitted in PR #9881.