Describe the bug
OxiCloud v0.6.0 OOMKilled (exit 137) when copying/uploading large files (~800 MB ISO) via WebDAV. The entire file content is buffered in memory during Content-Defined Chunking deduplication, even when the content already exists in the blob store (DEDUP HIT). This causes the container to exceed available memory on a node with 6 GiB RAM and 4 GiB memory limit, even with MIMALLOC_PURGE_DELAY=0 configured.
To Reproduce
- Deploy OxiCloud v0.6.0 in a Kubernetes pod with 4-5 GiB memory limit on a 6 GiB node
- Upload or copy an ~850 MiB ISO file via WebDAV (e.g., PUT /webdav/.../file.ISO)
- Observe pod OOMKilled within seconds
Expected behavior
Large files should be streamed through dedup without fully buffering in memory, especially when the content is already known (DEDUP HIT). The memory footprint during a copy operation should be proportional to the CDC chunk size, not the file size.
Actual behavior
The process reads the entire file into memory, consuming ~1-2+ GiB of RSS for an 850 MiB file, causing OOMKilled. Logs show the file is fully streamed through memory even on dedup hits:
DEDUP HIT (manifest): b758dff687e6 (855971840 bytes saved)
STREAMING WRITE: Office_2016_W32_Ru.ISO (855971840 bytes, hash: b758dff687e6)
Environment
- OxiCloud version: v0.6.0 (docker image
diocrafts/oxicloud:0.6.0, amd64)
- Deployment: Kubernetes (via raw manifests, not Helm)
- PostgreSQL: CloudNativePG 1.26
- Storage backend: local (NFS PV)
- Allocator: mimalloc with
MIMALLOC_PURGE_DELAY=0 and MIMALLOC_ALLOW_LARGE_OS_PAGES=0
Additional context
FileContentCache (moka): max_file=10MB, max_total=512MB, max_entries=10000 — cache limits are reasonable- Small files (<10 MB) work fine
- The CDC chunk store already contains the file chunks (10824 chunks, 511 manifests at startup), yet the streaming write still reads the full file
- Suggested areas to investigate:
dedup_service streaming write path, file_blob_write_repository buffering strategy in /webdav/ PUT handler
Describe the bug
OxiCloud v0.6.0 OOMKilled (exit 137) when copying/uploading large files (~800 MB ISO) via WebDAV. The entire file content is buffered in memory during Content-Defined Chunking deduplication, even when the content already exists in the blob store (DEDUP HIT). This causes the container to exceed available memory on a node with 6 GiB RAM and 4 GiB memory limit, even with
MIMALLOC_PURGE_DELAY=0configured.To Reproduce
Expected behavior
Large files should be streamed through dedup without fully buffering in memory, especially when the content is already known (DEDUP HIT). The memory footprint during a copy operation should be proportional to the CDC chunk size, not the file size.
Actual behavior
The process reads the entire file into memory, consuming ~1-2+ GiB of RSS for an 850 MiB file, causing OOMKilled. Logs show the file is fully streamed through memory even on dedup hits:
DEDUP HIT (manifest): b758dff687e6 (855971840 bytes saved)
STREAMING WRITE: Office_2016_W32_Ru.ISO (855971840 bytes, hash: b758dff687e6)
Environment
diocrafts/oxicloud:0.6.0, amd64)MIMALLOC_PURGE_DELAY=0andMIMALLOC_ALLOW_LARGE_OS_PAGES=0Additional context
FileContentCache (moka): max_file=10MB, max_total=512MB, max_entries=10000— cache limits are reasonablededup_servicestreaming write path,file_blob_write_repositorybuffering strategy in/webdav/PUT handler