fix: stop RAM growth during full pipeline runs - #868
Open
tqbdev wants to merge 2 commits into
Open
Conversation
tqbdev
requested review from
Map1en,
fffonion,
karrot0,
liksunrice and
mayocream
as code owners
July 21, 2026 03:14
Contributor
|
Thanks for your first PR to Koharu. Please review our contribution guide before review: In the PR description, include:
If AI helped produce the patch, a human still needs to review and understand it before submission. |
Processing all pages (detect/OCR/translate/inpaint/render) grew RAM monotonically and never released it. Three independent leaks: - Frontend: useBlobImage created object URLs that were never revoked, so every sprite/inpaint/render blob pinned a full decoded image in the webview for the whole run. Revoke URLs on query-cache eviction and drop the blob gcTime from 10min to 1min so inactive pages release promptly. - Metal: candle's Metal ops and the MPSGraph FFT allocate autoreleased Objective-C objects (command buffers, MPS intermediates), and the pipeline runs on tokio worker threads with no autorelease pool draining, so they accumulate for the entire run. Wrap the LaMa per-crop forward and the FFT calls in objc2 autorelease pools (no-op off Metal). - Unbounded caches: the LaMa FFT-plan caches (keyed by crop shape) and the flux2 Qwen prompt cache lived on whole-run engine instances and were never evicted. Bound them with LruCache. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tqbdev
force-pushed
the
fix/pipeline-ram-growth
branch
from
July 21, 2026 03:18
4426374 to
6b9b474
Compare
The autorelease-pool leak wasn't unique to LaMa: every candle Metal inference (detect, OCR, segmentation, font, AOT/flux2 inpaint) runs on pool-less tokio worker threads, so their autoreleased command buffers and MPS temporaries — and the GPU buffers those pin — accumulate across a full "process all pages" run the same way. Add a shared koharu_ml::autorelease_scope() helper (real objc2 pool on Metal, no-op otherwise) and wrap every candle-Metal engine's synchronous inference call with it, so each page's temporaries are freed as soon as that step returns. LaMa now uses the shared helper too. The llama.cpp engines (paddleocr-vl, translate) are unaffected — they use ggml's own Metal backend, not candle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Running the pipeline over all pages (detect → OCR → translate → inpaint → render) made RAM climb monotonically page after page and never release it, building to large consumption by the end of a batch. Reproduced on macOS with the LaMa inpainter on Metal.
Root causes
Three independent leaks (a fix for any one alone barely moved the needle):
useBlobImage(ui/hooks/useBlobData.ts) createdURL.createObjectURL(blob)but never revoked it, so every sprite/inpaint/render blob pinned a full decoded image in the webview for the whole run. Engine-independent, so backend changes didn't help it.lama/fft/metal.rs) allocate autoreleased objects (command buffers, MPS intermediates); a leaked command buffer also pins every GPU buffer it referenced. The pipeline runs on tokio worker threads that have no autorelease pool draining, so these accumulate for the entire run. There was noautoreleasepoolanywhere in the tree.lama/fft/{metal,cuda}.rs) and the flux2 Qwen prompt cache (flux2_klein/qwen.rs) lived on whole-run engine instances and were never evicted.Fixes
ui/lib/queryClient.ts: revoke ablobImagequery's object URL when it's evicted from the query cache;ui/hooks/useBlobData.ts: drop blobgcTime10min → 1min so inactive pages release promptly.lama/mod.rs+lama/fft/metal.rs: wrap the LaMa per-crop forward and the FFT calls inobjc2::rc::autoreleasepool(no-op off Metal via a small cfg-split helper).lama/fft/{metal,cuda}.rsandflux2_klein/qwen.rs: bound the caches withlru::LruCache.Verification
cargo check -p koharu-mlpasses with and without themetalfeature.cargo test -p koharu-ml --features metal --lib lama::passes.Notes
🤖 Generated with Claude Code