Skip to content

Fail model load early when free RAM is below the catalog estimate - #251

Merged
jatinkrmalik merged 8 commits into
mainfrom
fix/model-load-low-ram
Sep 5, 2026
Merged

Fail model load early when free RAM is below the catalog estimate#251
jatinkrmalik merged 8 commits into
mainfrom
fix/model-load-low-ram

Conversation

@jatinkrmalik

Copy link
Copy Markdown
Member

Summary

Fixes #250. Before WhisperKit/CoreML starts, refuse a load when installed RAM or os_proc_available_memory() is below that model's catalog estimate. The UI shows an error instead of spinning forever under memory pressure.

Test plan

  • Unit: SystemInfo.canFitModelInMemory physical / available / unknown / enough cases
  • Manual: pick a model larger than free RAM → error toast, spinner clears, previous model stays if one was loaded
  • Manual: enough free RAM → load still works

Known limits

  • Auto-select (size == nil) skips the gate until a concrete size is known
  • Catalog GB is an estimate; peak CoreML use can still be higher
  • No load timeout yet if the probe says OK but the engine still hangs

Before WhisperKit starts, refuse loads that cannot fit in installed
or available memory so the spinner does not hang (vocamac#250).
@netlify

netlify Bot commented Sep 4, 2026

Copy link
Copy Markdown

Deploy Preview for voca-mac canceled.

Name Link
🔨 Latest commit 94c7c65
🔍 Latest deploy log https://app.netlify.com/projects/voca-mac/deploys/6a9a7285e292d4000861a19c

@github-actions github-actions Bot added bug Something isn't working ci app and removed ci labels Sep 4, 2026
Swift imports the probe as Int; coerce non-positive readings to zero
so availableMemoryBytes matches its UInt64 return type.
os_proc_available_memory is unavailable on macOS and broke App CI.
Use free+inactive pages from HOST_VM_INFO64 instead; return 0 on probe failure.
CI free+inactive pages often sit under Medium's 5 GB estimate, so the new pre-load check rejected mock medium loads and broke serialization/restore tests.
@jatinkrmalik
jatinkrmalik marked this pull request as ready for review September 4, 2026 01:45
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds an early model-load memory check so known oversized models fail with an error before WhisperKit/CoreML begins loading.

  • Compares catalog RAM estimates with installed and currently available memory.
  • Preserves an already loaded model when the preflight rejects another load.
  • Uses conservative Darwin VM accounting and releases the acquired host send right.
  • Adds focused gate, state-preservation, and memory-threshold tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
Sources/VocaMac/Models/AppState.swift Adds the preflight refusal before loading-state mutations and leaves any active model untouched when memory is insufficient.
Sources/VocaMac/Services/SystemInfo.swift Adds installed- and available-memory capacity checks using free plus inactive Darwin pages with balanced Mach-port cleanup.
Tests/VocaMacTests/AppStateTests.swift Verifies that rejection avoids a Whisper load and preserves the previously loaded model and selection.
Tests/VocaMacTests/Mocks/MockServices.swift Makes test model loads deterministic by bypassing host-memory variability unless a test explicitly overrides the gate.
Tests/VocaMacTests/ModelTests.swift Covers insufficient physical memory, insufficient available memory, unavailable probe data, and sufficient capacity.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Request model load] --> B{Concrete model size known?}
    B -- No --> D[Continue existing load flow]
    B -- Yes --> C{Installed and available memory meet estimate?}
    C -- No --> E[Show error and preserve active model]
    C -- Yes --> D
    D --> F[Start WhisperKit/CoreML load]
Loading

Reviews (5): Last reviewed commit: "Use free+inactive only for the low-RAM p..." | Re-trigger Greptile

Comment thread Sources/VocaMac/Models/AppState.swift Outdated
Comment thread Sources/VocaMac/Services/SystemInfo.swift Outdated
Comment thread Sources/VocaMac/Services/SystemInfo.swift
@jatinkrmalik
jatinkrmalik marked this pull request as draft September 4, 2026 01:56
Refuse the load without restoring or clearing an already loaded
model. Count speculative, purgeable, and compressor pages in the
host probe, and release the mach_host_self send right.
Empty commit so Greptile re-scores the low-RAM gate fixes after the
prior review on 8108205. No code changes.
@jatinkrmalik

Copy link
Copy Markdown
Member Author

@greptileai please re-review the current head (low-RAM gate fixes in e47f66a + empty retrigger commit).

Comment thread Sources/VocaMac/Services/SystemInfo.swift Outdated
Do not add speculative pages on top of free_count (Darwin already
includes them), and drop compressor pages which still occupy RAM.
@jatinkrmalik

Copy link
Copy Markdown
Member Author

@greptileai please re-review eaf05b9 — VM accounting now uses free+inactive+purgeable only (no speculative double-count, no compressor).

Comment thread Sources/VocaMac/Services/SystemInfo.swift Outdated
Purgeable often overlaps the inactive queue on Darwin, so summing both
overstates reclaimable memory. Keep the gate conservative.
@jatinkrmalik

Copy link
Copy Markdown
Member Author

@greptileai please re-review 94c7c65 — available memory is free+inactive only (no speculative/purgeable/compressor overcount).

@jatinkrmalik
jatinkrmalik marked this pull request as ready for review September 4, 2026 07:34
@jatinkrmalik
jatinkrmalik merged commit 002a31c into main Sep 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Models keep loading infinitely even if there is not enough RAM!

1 participant