Skip to content

fix: stop RAM growth during full pipeline runs - #868

Open
tqbdev wants to merge 2 commits into
mayocream:mainfrom
tqbdev:fix/pipeline-ram-growth
Open

fix: stop RAM growth during full pipeline runs#868
tqbdev wants to merge 2 commits into
mayocream:mainfrom
tqbdev:fix/pipeline-ram-growth

Conversation

@tqbdev

@tqbdev tqbdev commented Jul 21, 2026

Copy link
Copy Markdown

Problem

Running the pipeline over all pages (detect → OCR → translate → inpaint → render) made RAM climb monotonically page after page and never release it, building to large consumption by the end of a batch. Reproduced on macOS with the LaMa inpainter on Metal.

Root causes

Three independent leaks (a fix for any one alone barely moved the needle):

  1. Frontend — object URLs never revoked. useBlobImage (ui/hooks/useBlobData.ts) created URL.createObjectURL(blob) but never revoked it, so every sprite/inpaint/render blob pinned a full decoded image in the webview for the whole run. Engine-independent, so backend changes didn't help it.
  2. Backend — no Objective-C autorelease pool on Metal. candle's Metal ops and the MPSGraph FFT (lama/fft/metal.rs) allocate autoreleased objects (command buffers, MPS intermediates); a leaked command buffer also pins every GPU buffer it referenced. The pipeline runs on tokio worker threads that have no autorelease pool draining, so these accumulate for the entire run. There was no autoreleasepool anywhere in the tree.
  3. Backend — unbounded ML caches. The LaMa FFT-plan caches (keyed by crop shape, lama/fft/{metal,cuda}.rs) and the flux2 Qwen prompt cache (flux2_klein/qwen.rs) lived on whole-run engine instances and were never evicted.

Fixes

  • ui/lib/queryClient.ts: revoke a blobImage query's object URL when it's evicted from the query cache; ui/hooks/useBlobData.ts: drop blob gcTime 10min → 1min so inactive pages release promptly.
  • lama/mod.rs + lama/fft/metal.rs: wrap the LaMa per-crop forward and the FFT calls in objc2::rc::autoreleasepool (no-op off Metal via a small cfg-split helper).
  • lama/fft/{metal,cuda}.rs and flux2_klein/qwen.rs: bound the caches with lru::LruCache.

Verification

  • cargo check -p koharu-ml passes with and without the metal feature.
  • cargo test -p koharu-ml --features metal --lib lama:: passes.
  • UI lint clean (no new warnings).
  • Manually confirmed on macOS/Metal: RAM now plateaus during "process all pages" instead of climbing to the end.

Notes

  • If growth ever shows up specifically during Detect/OCR (not Inpaint), it's the same candle-Metal autorelease issue in those engines and can be fixed with the same wrap.

🤖 Generated with Claude Code

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file area: ui labels Jul 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for your first PR to Koharu.

Please review our contribution guide before review:
https://koharu.rs/contribute/introduction/

In the PR description, include:

  • what changed
  • any user-visible behavior differences
  • how you verified the change

If AI helped produce the patch, a human still needs to review and understand it before submission.

Processing all pages (detect/OCR/translate/inpaint/render) grew RAM
monotonically and never released it. Three independent leaks:

- Frontend: useBlobImage created object URLs that were never revoked, so
  every sprite/inpaint/render blob pinned a full decoded image in the
  webview for the whole run. Revoke URLs on query-cache eviction and drop
  the blob gcTime from 10min to 1min so inactive pages release promptly.

- Metal: candle's Metal ops and the MPSGraph FFT allocate autoreleased
  Objective-C objects (command buffers, MPS intermediates), and the
  pipeline runs on tokio worker threads with no autorelease pool draining,
  so they accumulate for the entire run. Wrap the LaMa per-crop forward
  and the FFT calls in objc2 autorelease pools (no-op off Metal).

- Unbounded caches: the LaMa FFT-plan caches (keyed by crop shape) and the
  flux2 Qwen prompt cache lived on whole-run engine instances and were
  never evicted. Bound them with LruCache.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tqbdev
tqbdev force-pushed the fix/pipeline-ram-growth branch from 4426374 to 6b9b474 Compare July 21, 2026 03:18
The autorelease-pool leak wasn't unique to LaMa: every candle Metal
inference (detect, OCR, segmentation, font, AOT/flux2 inpaint) runs on
pool-less tokio worker threads, so their autoreleased command buffers and
MPS temporaries — and the GPU buffers those pin — accumulate across a
full "process all pages" run the same way.

Add a shared koharu_ml::autorelease_scope() helper (real objc2 pool on
Metal, no-op otherwise) and wrap every candle-Metal engine's synchronous
inference call with it, so each page's temporaries are freed as soon as
that step returns. LaMa now uses the shared helper too. The llama.cpp
engines (paddleocr-vl, translate) are unaffected — they use ggml's own
Metal backend, not candle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ui dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant