branch-4.1: [feature](file cache) Add asynchronous file cache writes - #67015
Merged
yiguolei merged 1 commit intoAug 24, 2026
Merged
Conversation
Problem Summary:
On a file-cache miss, `CachedRemoteFileReader` currently performs two
different kinds of work on the query thread:
1. read bytes from remote storage to satisfy the query;
2. append and finalize those bytes into the local file cache for future
queries.
The first operation is on the critical path. The second is best-effort
cache population, but local filesystem latency and backpressure are
still charged to foreground scan latency because the two operations are
coupled.
This PR implements phase 1 of asynchronous file-cache writes. For
ordinary reads, the query thread returns after the caller's buffer is
complete and hands eligible cache-miss blocks to per-disk background
workers. The feature is opt-in and disabled by default. Explicit
cache-population operations, including warm-up and prefetch/download
paths, retain synchronous completion semantics.
Simply moving `append()` to another thread is unsafe. The design also
has to preserve cache-block ownership, avoid duplicate queued writes,
survive clear/remove races, bound background work, and keep existing
warm-up and dry-run behavior. This PR introduces dedicated components
for those responsibilities instead of transferring existing `FileBlock`
objects across threads.
- Remove best-effort local cache persistence from the ordinary
remote-read critical path.
- Reuse remote bytes that have already been fetched but are still
waiting for local persistence.
- Avoid duplicate background writes for the same cache block.
- Bound the buffer-capacity ownership of accepted queued and active
tasks without making queries wait for disk persistence; when saturated,
prefer newly read blocks by evicting only the oldest queued task.
- Prevent stale tasks from recreating cache data after clear/remove
operations.
- Preserve the existing synchronous behavior for explicit
cache-population callers.
- Provide query-profile, per-cache-disk core, and BE-global cache-probe
observability for the new path.
- This is not remote-read singleflight. Two cold readers that overlap
before either publishes its completed remote buffer can still issue
separate remote reads.
- The asynchronous write path does not consume peer-cache reads. A call
that selects peer cache keeps synchronous cache-write semantics; after a
call selects asynchronous write, it pins direct remote storage for the
rest of that call even if the online peer-read policy changes.
- This does not change local file-cache durability guarantees;
persistence remains best effort.
- This does not make warm-up or prefetch completion asynchronous.
- The asynchronous path does not support `file_cache_query_limit_bytes`
or `file_cache_query_limit_percent`; deployments that depend on either
query-scoped admission limit must keep asynchronous file-cache writes
disabled.
- This does not enable the feature by default.
A storage-compute separated PL1 benchmark was run on commit
`b586dd1f429` with one 32-core, 123 GiB BE, two PL1 file-cache disks, 16
background write workers per disk, and `enable_adaptive_scan=false`.
Each round restarted the BE with an empty file cache. The workload was a
full scan of `ssb.lineorder`: about 6 billion rows and 17 columns, with
135.8 GB reported by `SHOW DATA`.
At 64 target scanners, the estimated active buffer working set was `16
instances x 4 scanners x 17 columns x 1 MiB = 1,088 MiB`, or about 544
MiB per cache disk.
| Mode | Effective pending ownership per cache instance | Query time |
Remote I/O requests |
| --- | ---: | ---: | ---: |
| Sync | 0 | 171 s | 138,290 |
| Async | 256 MiB | 136 s | 261,980 |
| Async | 512 MiB | 104 s | 186,861 |
| Async | 1,024 MiB | 90 s | 150,509 |
| Async | 2,048 MiB | 89 s | 144,457 |
With the pending ownership limit below the working set, older queued
buffers can be evicted before later scans reuse them or before
persistence completes, causing repeated remote reads. Once the pending
ownership limit covers the working set, reuse improves and foreground
scans no longer wait for local persistence. At 1,024 MiB per disk, query
time fell from 171 s to 90 s, a 47.4% reduction. Increasing the limit to
2,048 MiB per disk improved the result by only one additional second.
The synchronous baseline already drove one cache disk to 94.4%
utilization, confirming that local cache writes were the bottleneck in
this workload.
The pending ownership limit was then increased with scanner concurrency
so that each run continued to cover its estimated working set. Remote
I/O stayed within 6.2% of the synchronous cold-cache baseline.
| Peak active scanners | Effective pending ownership per cache instance
| Query time | Network receive | Combined disk write | Remote I/O delta
|
| ---: | ---: | ---: | ---: | ---: | ---: |
| 63 | 2,048 MiB | 90 s | 1,677.4 MiB/s | 634.6 MiB/s | +4.4% |
| 94 | 3,072 MiB | 64 s | 2,398.7 MiB/s | 634.5 MiB/s | +6.2% |
| 128 | 4,096 MiB | 56 s | 2,725.8 MiB/s | 635.8 MiB/s | +5.6% |
As peak scanner concurrency increased from 63 to 128, query time fell
from 90 s to 56 s and network receive throughput increased, while
combined cache-disk write throughput remained approximately 635 MiB/s.
This shows that asynchronous persistence removed the saturated local
disks from the foreground critical path for this workload. Gains began
to narrow at 128 scanners, indicating that the next bottleneck was
shifting toward the network, remote storage, or CPU.
These results are workload-specific rather than a universal speedup
claim. The benefit requires both a foreground bottleneck in local
file-cache writes and enough effective accepted-task ownership on each
cache instance to cover the active scanner working set. The tables
report the effective per-instance shares used by the benchmark. With the
current configuration, `async_file_cache_write_max_pending_bytes` is one
BE-wide total divided equally among successfully initialized cache
instances; reproducing a two-disk row therefore requires a BE-wide total
twice the listed per-instance value. This remains an ownership limit,
not an aggregate hard limit on all live async buffer memory.
The query thread owns read planning and result assembly. It performs one
whole-range inflight lookup and, unless inflight buffers cover every
aligned block, at most one read-only probe for the complete aligned
request. It then represents each aligned block with a small source
classification. Background workers own cache-cell acquisition and
persistence. The two sides exchange immutable, reference-counted buffers
through an inflight index and a per-disk manager whose admission is
bounded by accepted queued and active task ownership.
```mermaid
flowchart LR
Caller[Scan / query caller]
Reader[CachedRemoteFileReader]
Remote[Direct remote FileReader]
subgraph CacheDisk[One file-cache disk]
Probe[BlockFileCache read-only probe]
Index[InflightWriteBufferIndex]
Manager[AsyncCacheWriteManager]
Queue[Locked FIFO deque with byte admission]
Workers[Resizable write workers]
Storage[BlockFileCache and local storage]
end
Metrics[Runtime profile, per-disk bvars, and global probe bvars]
Caller -->|read request| Reader
Reader -->|lookup completed remote buffers| Index
Reader -->|observe cache state without ownership| Probe
Reader -->|one remaining middle-range read| Remote
Remote -->|aligned bytes| Reader
Reader -->|complete caller buffer| Caller
Reader -->|publish true MISS buffers| Index
Reader -->|submit without disk IO wait| Manager
Manager --> Queue
Queue --> Workers
Workers -->|revalidate, acquire, append, finalize| Storage
Workers -->|conditional cleanup| Index
Storage -.->|clear/remove advances write epoch| Manager
Reader -.-> Metrics
Index -.-> Metrics
Manager -.-> Metrics
```
| Component | Responsibility | Important guarantee |
| --- | --- | --- |
| `CachedRemoteFileReader` | Resolves synchronous versus asynchronous
mode for every read, pins asynchronous calls to direct remote storage,
performs one whole-range inflight lookup and at most one whole-range
cache probe, classifies aligned blocks, performs the required remote
read, fills the caller buffer, and submits only true misses. | The
caller never observes an incomplete buffer. Cache-write submission is
not allowed to turn a successful remote read into a query failure. |
| `BlockFileCache::probe` | Reports downloaded, downloading, empty,
deleting, and missing ranges without creating cache cells or taking
downloader ownership. | Planning does not mutate cache state or transfer
thread-affine `FileBlock` downloader ownership. LRU touch happens only
after a block is actually consumed. |
| `InflightWriteBufferIndex` | Stores shard-protected references to
completed remote buffers that are pending persistence, keyed by cache
key and block offset and tagged with a write epoch. | Insert-if-absent
selects one background-write owner. Conditional removal prevents an old
completion callback from removing a newer entry. |
| `AsyncCacheWriteManager` | Owns the per-disk mutex-protected FIFO
`deque`, queued/active/pending count and byte accounting, fixed
drop-oldest admission, tracked-buffer accounting, resizable workers,
shutdown, and manager-level metrics. | Submission does not wait for disk
I/O. At the byte limit it accepts the new fixed-size task by evicting
only the oldest queued task; if there is no queued victim or one task
exceeds the limit, it rejects and rolls back the new inflight entry.
Active tasks are never evicted. |
| Async write worker | Rechecks the write epoch and current cache state,
calls the normal cache admission/cell-creation path, skips blocks
already downloaded or owned by another downloader, and appends/finalizes
only eligible empty blocks. | Background work follows current cache
state; it does not blindly overwrite or resurrect blocks. |
| Cache write epoch | Represents the generation of work accepted by a
cache instance. Clear/remove paths advance it. | A task accepted under
an older generation is dropped before or during persistence. |
| Profile and bvar integration | Records query-local
submission/rejection/inflight reuse, five core per-cache-disk bvars, and
four BE-global probe counters. | Operators can observe query-path
outcomes, worker availability, live async-buffer memory, FIFO eviction,
and cache-probe results without multiplying low-value per-disk series. |
Each aligned block is classified in this order:
1. completed remote buffers in `InflightWriteBufferIndex`;
2. existing cache blocks through the read-only probe;
3. remaining blocks as misses.
The lookup covers the complete aligned request. A full inflight hit
skips the file-cache probe; otherwise one probe covers the same whole
range, so the planner does not build or repeatedly probe individual
uncovered runs. It identifies the first and last real cache miss,
materializes only inflight/cache blocks outside those boundaries, and
retains the existing wait behavior for a `DOWNLOADING` block outside the
remote span. If misses remain, everything from the first through the
last miss is fetched with one aligned direct-remote read. Existing
cache, inflight, or downloading blocks inside that span are
intentionally covered by the same remote result instead of being filled
or waited one by one. Only the blocks that were real misses are
published and submitted for asynchronous persistence.
```mermaid
sequenceDiagram
participant C as Caller
participant R as CachedRemoteFileReader
participant I as Inflight index
participant F as BlockFileCache
participant O as Direct remote storage
participant S as Async write manager
participant W as Worker
C->>R: read_at(offset, size)
R->>I: lookup aligned blocks for current epoch
I-->>R: reusable pending buffers
R->>F: at most one read-only probe for the aligned request
F-->>R: downloaded / downloading / miss coverage
R->>R: materialize blocks outside the first-to-last miss span
alt all requested bytes are covered
R-->>C: return completed caller buffer
else a middle range remains
R->>O: one aligned middle-range read
O-->>R: remote bytes
R->>R: copy requested overlap into caller buffer
loop each true MISS block in the remote span
R->>I: insert completed block buffer if absent
alt another owner already published it
I-->>R: existing entry, skip duplicate write
else this reader owns persistence
R->>S: try_submit without waiting
alt fits pending byte limit
S-->>R: accepted at FIFO tail
else full with a queued victim
S-->>R: accepted and oldest queued task evicted
S->>I: conditional victim cleanup outside queue lock
else task exceeds byte limit, no queued victim, or manager stopped
S-->>R: rejected
R->>I: conditional rollback
end
end
end
R-->>C: return completed caller buffer
S->>W: dequeue in background
W->>F: revalidate epoch and current block state
W->>F: append and finalize eligible empty blocks
W->>I: conditional completion cleanup
end
```
After the first remote read publishes a buffer and before its worker
finishes persistence, a later reader can copy the requested bytes
directly from the inflight entry. It does not wait for the local file to
be finalized and does not submit another write task for that block.
This index deduplicates pending cache writes and bridges the interval
between remote-read completion and local persistence. It intentionally
does not claim to deduplicate remote requests that began before
publication.
Each cache instance's share of the BE-wide pending-byte limit accounts
the full buffer-capacity bytes owned by accepted queued and active
tasks. The manager protects a FIFO `deque` and all queued/active/pending
ownership transitions with one mutex, maintaining both `pending_count =
queued_count + active_count` and `pending_bytes = queued_bytes +
active_bytes`.
This is an admission and ownership-accounting limit, not a hard upper
bound on every live async buffer allocation. A buffer is allocated
before `try_submit()`. After a replacement or worker completion, the
task can also remain live while its callback removes the inflight entry
and while other shared references are released, even though it no longer
contributes to `pending_bytes`. The manager memory tracker and
inflight-index buffer gauges expose those live allocations separately.
Production submits one task per aligned cache block. Every task
allocates exactly `file_cache_each_block_size` bytes; `write_size` is
the valid prefix and may be shorter only for the physical EOF block.
Admission and byte accounting use the full buffer capacity, including
for that short tail block. Therefore a configured remainder smaller than
one cache block is intentionally unusable, and replacing one queued task
with another preserves pending bytes exactly.
- If one task's fixed buffer is larger than the effective per-instance
limit, the task is rejected.
- While `pending_bytes + task_buffer_bytes` fits, the new task is
appended to the FIFO tail and increases queued and pending count/byte
state.
- Without capacity but with at least one queued task, the manager
appends the new task and removes exactly the oldest queued task from the
head. The new submission succeeds; queued and pending count/byte state
stays unchanged, and active tasks are never selected as victims.
- Without capacity and with an empty queue, every pending task is
active, so the new task is rejected because there is no queued victim.
- After a runtime limit decrease, replacement continues while a queued
victim exists even if current pending bytes remain above the new limit.
This does not increase accounted pending ownership; active work drains
toward the new limit.
Evicted-task metrics, callback execution, conditional inflight-index
removal, and buffer release happen outside the queue mutex. An evicted
task is best-effort cache population that never reaches a worker; it
does not affect the query that already received its remote bytes.
Likewise, an append/finalize failure is logged and counted by the
worker; it does not retroactively fail a query whose remote read
succeeded. A future read simply observes another cache miss and may
retry population.
Cache invalidation advances the manager's write epoch. Both inflight
lookup/replacement and worker processing compare their entry or task
against the current epoch. Workers check the epoch before acquiring
cache blocks and again while processing them. This closes the window
where old queued work could recreate blocks after a clear/remove
operation.
Completion cleanup uses pointer-conditional removal. Therefore, even if
a newer generation has published an entry for the same block offset, an
older task cannot erase it.
`CacheWriteMode` is resolved for every read so an online configuration
switch affects existing readers. Ordinary reads may select the
asynchronous path when the feature is enabled. Peer-cache reads are not
supported by the asynchronous path: a read that observes peer mode
during resolution selects synchronous cache write. Once asynchronous
mode has been selected, the reader copies the call's `IOContext` and
sets `bypass_peer_read` on that private copy, pinning direct remote
storage for the rest of the call even if the online peer-read policy
changes; the caller's context is not mutated. Dry-run, warm-up,
segment-index population, downloader, and explicit prefetch paths also
select or override synchronous mode because their completion semantics
depend on the cache being populated before returning.
The default remains synchronous because `enable_async_file_cache_write`
defaults to `false`.
The asynchronous path currently does not support
`file_cache_query_limit_bytes` or `file_cache_query_limit_percent`.
Background tasks do not retain the query-owned
`RemoteScanCacheWriteLimiter` or the query-context-holder lifetime
required by those admission policies, so their limits are not guaranteed
for asynchronous persistence. Keep asynchronous file-cache writes
disabled when either query-level limit is required.
Each `BlockFileCache` creates its inflight index and async-write manager
state during initialization. When asynchronous writeback is disabled,
the manager does not create persistent worker loops. Enabling the
feature online explicitly starts all initialized per-disk managers.
Workers pop one task at a time from the FIFO head under the same queue
mutex used by producers and atomically move it from queued to active.
There is no batch-size setting or bulk dequeue. When an active task
finishes, active and pending count/byte state is decremented under the
queue mutex; lifecycle metrics and the task callback run after releasing
it.
Each worker object owns its stop request and completion signal. Worker
count can be resized online without exposing numeric worker IDs as
lifecycle state. Before submitting long-running Worker tasks, the
manager raises the pool's minimum thread count to the configured worker
count, so an OS-thread creation failure is returned before any new
Worker task can be accepted and stranded in the pool queue. Shutdown
first stops new submitters, waits for already registered submitters,
drains accepted tasks, joins workers, and only then destroys state
referenced by task-finalization callbacks. There is no second
persistence pool or cross-thread `FileBlock` ownership transfer.
| Configuration | Default | Purpose |
| --- | ---: | --- |
| `enable_async_file_cache_write` | `false` | Enables the ordinary
asynchronous write path. |
| `async_file_cache_write_workers_per_disk` | `16` | Number of
persistence workers for each cache disk; supports online resize. Workers
are started only when asynchronous writeback is enabled. |
| `async_file_cache_write_max_pending_bytes` | `-1` (automatic) |
BE-wide accepted queued + active task buffer-capacity ownership limit. A
positive value is the exact BE-wide total; `-1` resolves to `max(1 GiB,
1% of the BE memory limit)` before the total is split equally across
successfully initialized cache instances. Supports online update. |
| `enable_async_file_cache_write_inflight_write_buffer_index` | `true` |
Enables pending-buffer reuse and duplicate-write suppression. |
| `async_file_cache_write_inflight_write_buffer_index_shard_count` |
`64` | Controls index lock sharding at cache initialization; it is not
mutable online. |
`async_file_cache_write_max_pending_bytes` is one BE-wide total. After
cache initialization, `FileCacheFactory` divides it by the number of
successfully initialized cache instances and applies the same
floor-divided share to every per-disk manager. Cache keys are
hash-sharded across those instances, so this keeps the aggregate
ownership budget stable as the cache-disk count changes;
integer-division and sub-cache-block remainders are intentionally
unused. The value is an upper bound on accepted queued + active task
ownership, not a reservation or a hard upper bound on all live async
buffer memory. Task buffers are allocated lazily before admission, and
finalizing or externally referenced buffers can outlive pending
ownership. The queue-full behavior is fixed to FIFO drop-oldest and has
no separate policy configuration. Accepted tasks are not discarded by an
age watchdog.
Query runtime profiles expose asynchronous submissions, rejections,
buffer-allocation failures, stale-epoch drops, and inflight hits/misses.
Each `BlockFileCache` instance exposes five bvars prefixed by its cache
base path:
- `async_cache_write_running_worker_count`: currently running
persistence workers;
- `async_cache_write_buffer_memory_bytes`: all live
`AsyncCacheWriteBuffer` capacity charged to the manager memory tracker;
- `async_cache_write_submitted_task_count`: accepted asynchronous write
tasks;
- `async_cache_write_evicted_oldest_task_count`: queued tasks evicted by
fixed FIFO drop-oldest admission;
- `inflight_write_buffer_index_buffer_bytes`: buffer capacity currently
retained by inflight-index entries.
The BE exposes four global cache-probe bvars:
- `cached_remote_file_reader_probe_count`: whole-range cache probes
performed while building asynchronous read plans;
- `cached_remote_file_reader_probe_hit_downloaded_count`: probed logical
blocks already downloaded;
- `cached_remote_file_reader_probe_hit_downloading_count`: probed
logical blocks currently downloading;
- `cached_remote_file_reader_probe_miss_count`: probed logical blocks
classified as real misses.
Detailed queue and pending accounting, rejection-reason breakdowns,
operation latencies, write-epoch details, state-based skips, failures,
and inflight-index operation breakdowns are not exposed as named bvars.
Query-relevant outcomes remain available through runtime profiles, while
focused tests inspect manager diagnostics directly without adding scrape
series.
`async_cache_write_buffer_memory_bytes` is the more complete live-memory
signal: task buffers are allocated before admission and can outlive
pending ownership while callbacks and other shared references release
them. `inflight_write_buffer_index_buffer_bytes` reports only capacity
retained by index entries, so the two gauges can differ transiently and
when the index is disabled.
- Latest bvar-surface reduction: `./build.sh --be -j100`,
`build-support/clang-format.sh`, and `build-support/check-format.sh`
passed. Focused unit tests were not rerun because the final commits only
removed metric exposure and collection; the behavioral tests below
predate that observability-only cleanup.
- Latest review fixes: **7/7 targeted ASAN BE unit tests passed**.
- asynchronous reads remain pinned to direct remote storage after an
online peer-policy change;
- tracked-buffer allocator failures are caught and converted to a
status;
- worker resize reserves backing pool threads before publishing
long-running Worker tasks, while existing stop/resize ownership tests
continue to pass.
- Current locked-FIFO/drop-oldest manager: **21/21
`AsyncCacheWriteManagerTest` cases passed** after the worker-lifecycle
encapsulation.
- FIFO order, active-only saturation, one-for-one oldest eviction,
epoch-safe callback cleanup, runtime byte-limit convergence, direct
count/byte conservation, worker resize, and shutdown/replacement races.
- Current one-task-at-a-time queue and read integration: **36/36 focused
manager, inflight-index, and async-reader tests passed** after removing
the ineffective batch-size path.
- Pending-ownership-bounded admission and read/task contracts: **37/37
focused manager and reader tests passed**.
- fixed cache-block buffer capacity with a short physical EOF
`write_size`;
- queued + active byte limits, one-for-one oldest replacement, runtime
limit decrease, inflight cleanup, and direct count/byte accounting.
- Latest BE-wide pending-limit update: **26/26 focused manager and
configuration BE unit tests passed**.
- positive BE-wide totals, the 1 GiB automatic floor, 1% memory scaling,
and invalid sentinels;
- one- and two-cache creation, equal per-instance splitting, and online
switching between a fixed BE-wide total and `-1` automatic sizing.
- Latest read-planning refactor: **6/6 targeted ASAN
`BlockFileCacheTest` cases passed**.
- inflight-buffer reuse and downloaded-block reads;
- `DOWNLOADING` wait outside the remote span;
- cached prefix/suffix assembly;
- one first-to-last-miss remote read with an existing middle block;
- true-miss-only submission and backpressure rollback;
- per-read cache-write mode resolution.
- Docker cloud regression: **1/1 suite passed**.
- a cold query submits asynchronous cache writes;
- after the queue drains, a second query observes probe hits;
- switching the feature off restores synchronous behavior without
asynchronous submissions.
- Build and style:
- `./build.sh --be --fe --cloud -j100`
- `./build.sh --be -j100`
- `./build.sh --be --file-cache-microbench -j100` and an all-mode
microbenchmark smoke run completed without hanging
- `build-support/check-format.sh`
- clang-tidy was intentionally not run for this change.
Add an opt-in asynchronous file-cache write path controlled by
`enable_async_file_cache_write`. Each cache disk uses a locked FIFO
whose accepted queued + active task buffer ownership is byte-bounded;
when the effective per-instance limit is reached, a new fixed-block task
replaces the oldest queued task, while active tasks are never evicted.
This is not a hard upper bound on every live async buffer allocation.
`async_file_cache_write_max_pending_bytes` defaults to `-1`, which
resolves to a BE-wide `max(1 GiB, 1% of the BE memory limit)`; the
resolved total is split equally across successfully initialized cache
instances. The asynchronous path does not support peer-cache reads,
`file_cache_query_limit_bytes`, or `file_cache_query_limit_percent`. The
feature remains disabled by default.
(cherry picked from commit be88cea)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
bobhan1
changed the title
bobhan1
marked this pull request as ready for review
August 21, 2026 03:05
Contributor
Author
|
run buildall |
Contributor
Author
|
run p0 |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run p0 |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
yiguolei
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Related PR: #65658
Problem Summary:
Backport #65658 to
branch-4.1, adding opt-in asynchronous file-cache writes while keeping synchronous writes as the default behavior.Branch-specific conflict resolution keeps the
branch-4.1CMake/source layout, forces file-cache downloader writes to remain synchronous, and adds a focused downloader test because the broader upstream fixture is not present on this branch.Release note
None
Check List (For Author)
Validation:
./build.sh --be -j100./run-be-ut.sh --run --filter='AsyncCacheWriteManagerTest.*:InflightWriteBufferIndexTest.*:BlockFileCacheProbeTest.*:FileCacheProfileReporterTest.*:FileCacheBlockDownloaderTest.*:CachedRemoteFileReaderTest.*' -j100(39/39 passed)build-support/check-format.shgit diff --check upstream/branch-4.1...HEADThe added cloud regression suite was not run locally.
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)