HDDS-15925. Skip redundant InfoBucket RPC on OFS getFileStatus with a bounded bucket-layout cache

[yandrey321]

What changes were proposed in this pull request?

Problem

On rooted OFS (ofs://), a getFileStatus on a key issues two OM RPCs: getBucket (InfoBucket, to fetch and validate the bucket layout) followed by the actual getFileStatus. This doubles client→OM traffic on getFileStatus-heavy workloads (e.g. Hive/Impala stats collection, listing-then-stat patterns), and the InfoBucket calls are the single largest caller class on the OM read path under load.

The layout fetch exists for one reason on the read path: to reject OBJECT_STORE (OBS) buckets client-side, because OM itself does not reject getFileStatus on an OBS bucket. So the RPC can't simply be dropped without changing behavior.

Approach

Add a bounded, configurable client-side cache of resolved (volume, bucket) → BucketLayout. Bucket layout is immutable for a bucket's lifetime, so caching it is safe; a TTL bounds the only edge case (delete + recreate with a different layout).

On a non-snapshot key getFileStatus:

Atomic get-or-load the resolved layout via Guava Cache.get(key, Callable) — a single thread resolves a given bucket on a miss while others wait, so concurrent cold misses don't each fire an InfoBucket RPC.
Re-validate the cached layout on every call with OzoneFSUtils.validateBucketLayout, which throws IllegalArgumentException for OBS exactly as getBucket does today — whether the layout came from the cache or a fresh fetch.
The cache holds every resolved layout, OBS included, so repeated access to an OBS bucket is rejected from the cache with zero further RPCs. OMException mapping (FILE_NOT_FOUND / BUCKET_NOT_FOUND → FileNotFoundException) is preserved by rethrowing the original IOException out of the loader. Snapshot paths and all write paths keep calling getBucket unchanged.

Net effect (per bucket, within TTL): first call = 2 RPCs; subsequent FS-bucket calls = 1; subsequent OBS calls = 0 (rejected from cache). getFileStatus behavior is identical to master, only cheaper.

Configuration

ozone.client.fs.bucket.layout.cache.expiry — ConfigType.TIME, default 2m. Bounds staleness of a cached layout.
ozone.client.fs.bucket.layout.cache.size — default 1000. Bounds cache memory; 0 disables caching (every call re-fetches and re-validates, still correct).
Testing

Generated-by: Claude Code (claude-opus-4-8)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15925

How was this patch tested?

1. CI: https://github.com/yandrey321/ozone/actions/runs/33423535153/job/99591663437
2. unit test, intgration tests
3. Benchmark (TestOfsGetFileStatusCacheBenchmark, tagged @tag("benchmark") One MiniOzoneCluster, 200 FSO buckets, 2000 getFileStatus calls (90% cache-hit workload), measured single-threaded and across 10 concurrent client threads sharing one FileSystem. Numbers below are baseline (master) vs this PR, on the same host.

Single-threaded

Metric	Baseline	This PR	Change
InfoBucket RPCs	2000 (1.00/call)	200 (0.10/call)	−90%
getFileStatus RPCs	2000	2000	unchanged
latency mean	0.266 ms	0.172 ms	−35%
latency p99	0.512 ms	0.377 ms	−26%
throughput	3,748.7 ops/s	5,786.8 ops/s	+54%

10 concurrent threads

Metric	Baseline	This PR	Change
InfoBucket RPCs	2000 (1.00/call)	200 (0.10/call)	−90%
getFileStatus RPCs	2000	2000	unchanged
latency mean	0.615 ms	0.403 ms	−34%
throughput	16,086.3 ops/s	24,439.7 ops/s	+52%

The −90% InfoBucket RPC reduction holds identically at 1 and 10 threads: the atomic get-or-load keeps the count at one per bucket even under concurrency, rather than stampeding to one per call. getFileStatus RPC count is unchanged, confirming behavior is preserved