Add option to disable footer caching to allow for large stream size without causing OOM in fadvise=RANDOM

[Animeshz]

Context

Fixes #1734.

The connector by default assumes the file being read with fadvise=RANDOM is a columnar format, which has metadata like schema, row-index, etc. at the footer. Hence caches the footer in memory, but this becomes an issue if

• File is not a columnar format
• fs.gs.inputstream.min.range.request.size is a big size, such as 2GB, it'll allocate byte[] of size 2GB per thread (or total read-channels opened) which reads the footer.

Therefore causing OOM.

Changes

Introduces option fs.gs.inputstream.footer.cache.enable (default: true) to allow disabling the footer cache via the configuration option when not required.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gemini-code-assist]

Code Review

This pull request introduces the fs.gs.inputstream.footer.cache.enable configuration option to allow disabling the in-memory footer cache for HTTP and gRPC read channels, preventing large memory allocations when the minimum range request size is large. The review feedback highlights two efficiency issues where the connector still requests and downloads the entire footer range from GCS even when the footer cache is disabled in GoogleCloudStorageReadChannel and GoogleCloudStorageClientReadChannel. Additionally, a potential log flooding issue was identified in GoogleCloudStorageBidiReadChannel due to an INFO log being triggered on every read call when the cache is disabled, which should be lowered to a finer log level.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud