system: mount SD cards async with bounded dirty-page writeback

[WLTB-Gino]

Problem

SD cards are automounted with -o sync (/usr/lib/mdev/automount). With a sync mount, every write(2) blocks until the SD card acknowledges it — only one write in flight per writer. That keeps RAM pressure down, but it means card-internal GC pauses (100 ms to over a second) propagate directly into whatever is writing: raptor/rmr drops frames while recording, timelapse stalls.

Change

• Drop sync from the automount, keep noatime — this is the dominant change. sync is far more aggressive than anything else in this PR: zero buffering, every write blocks immediately, every card stall lands directly in the writer.
• Bound dirty pages globally in /etc/sysctl.conf (applied by S00sysctl):
  • vm.dirty_background_bytes = 512K — flusher wakes early, keeps steady-state dirty memory near zero
  • vm.dirty_bytes = 4M — hard throttle ceiling; the remaining headroom absorbs card stalls
• sync on umount — with an async mount, the lazy umount could otherwise leave up to 4 MB unflushed when the card is pulled.

RAM is still capped — 4 MB of dirty pages replaces "one write in flight" as the bound — but writes coalesce and GC stalls are absorbed by the dirty budget instead of stalling the recorder.

Before / after

Before: sync mount. Zero dirty buffering on the card — writes block on every write(2), so card GC pauses hit the recorder in full.

After: async mount with pinned byte limits. The kernel default would be ratio-based (dirty_background_ratio = 10 / dirty_ratio = 20 — ~4.3 MB / ~8.6 MB on a 43 MB board, scaling with RAM); this PR pins deterministic bytes instead so the limits are identical on every board. Compared to the sync mount, the write path is massively looser — up to 4 MB of buffering where before there was none.

Testing

Not yet tested on hardware — validation on a real recording workload (raptor/rmr to SD) is pending. Feedback from testers on other SoC families and workloads welcome.

[themactep]

Thanks for this — the direction is right, but I'm not comfortable merging it as-is. This is a durability change, not a perf change, and it's untested on hardware. Two things need addressing first.

1. This needs before/after hardware test results, not a design argument.

Dropping sync changes what happens to footage on power cut / card yank. The SD cards are FAT32/exFAT (formatsd uses mkfs.vfat/mkfs.exfat) — non-journaled. sync has been on this mount since before Oct 2024 (it predates the noatime commit in 9c4c51f); it wasn't accidental, it's yank-resistance. With async, a card pulled without unmount loses up to the dirty budget (4MB) and can leave FAT/directory metadata inconsistent. That may be an acceptable trade for a recorder dropping frames — but it's a product decision, and it has to be demonstrated, not assumed.

Please provide real hardware results comparing sync vs. this patch on an actual recording workload, specifically:

• Frame drops / recording stalls during a forced SD GC stall (the claimed benefit — show the sync mount dropping frames and the async mount not). A measurable before/after, not a theory.
• No corruption after: (a) graceful unmount, (b) card yank mid-recording, (c) power cut mid-recording. Re-mount and fsck after each.
• On a 64MB T31 (the tightest board) with raptor running — confirm the 4MB dirty ceiling doesn't add memory pressure / OOM.
• Confirm sysctl -q -p applies cleanly on all three live kernels (3.10.14, 4.4.94, 7.1-rc1).

2. Minor: the sysctl comment is misleading.

vm.dirty_bytes/vm.dirty_background_bytes are global, not SD-specific — they also govern USB mass storage and anything else on the generic writeback path. The comment says "Keep SD card writeback asynchronous" — reword to state it's a global bound.

Also worth stating in the commit message: the sync in do_umount does nothing for the "user yanked the card" case — by then the device is gone and writeback just errors out. The 4MB bound is the only protection in that scenario. That tradeoff should be a conscious, documented decision, not an implied one.

[nicolasroche898]

first test results on a camera running record+audio+timelapse+stream (stress test):

PR #1535 works, and clearly. Frame drops fell 4.9× on the Jooan A6M (T23/64MB)
  
  ┌──────────────────────────┬────────────────────────────┬────────────────────────┐
  │                          │ BEFORE (-o sync)           │ AFTER (async, 512K/4M) │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ n                        │ 720 segments               │ 13 segments            │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ mean frames              │ 4428.2 — 1.60% dropped     │ 4485.3 — 0.33% dropped │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ median                   │ 4447                       │ 4488                   │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ min / max                │ 3219 / 4493                │ 4457 / 4500            │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ perfect segments (=4500) │ 0 of 720                   │ 2 of 13                │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ lost >1%                 │ 435 (60.4%)                │ 0 (0.0%)               │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ lost >3%                 │ 87 (12.1%)                 │ 0                      │
  ├──────────────────────────┼────────────────────────────┼────────────────────────┤
  │ lost >5%                 │ 6 (0.8%)                   │ 0                      │
  └──────────────────────────┴────────────────────────────┴────────────────────────┘
  
  Zero of 13 segments losing >1%, against a 60.4% base rate, is p ≈ 5.9 × 10⁻⁶ by chance. And two
  segments hit exactly 4500 — a perfect five minutes, which never once happened in 720 before-samples
  where the best was 4493.

Further tests from other users are welcome. I dont have a T31/64MB to test.
Related questions:

1. when to fsync() in raptor (currently: on file close)
2. whether to run with -o fsync (not -o sync)
3. whether to define dirty budget as a percentage, as it is currently in thingino (background 10 percent / dirty 20 percent of total RAM size) or as MBs (512k/4MB), as is defined in this PR.
4. whether the "yank test" is a valid requirement, and if so, what is the tolerated loss and how is it defined? 1 video file? 1 frame? 4MB? low probability of general file/dir corruption? is 1 video file loss acceptable (raptor fsync on fclose() )?

Note that the patch increases the total free RAM, as it decreases the existing dirty budget from 8.6MB (20%) to 4MB on jooan a6m, and at the same time improves the reliability and SIGNIFICANTLY decreases the frame loss. win win

[WLTB-Gino]

Companion work on the streamer side (concept stage, being thought through): gtxaspec/raptor#46.

This PR absorbs card GC pauses with a bounded dirty-page budget; the raptor PR targets the other failure surface: power loss and card yank. As the streamer streams video into a recording file, that growing file keeps a FAT sector hot for the entire recording: each time the file grows past its current allocation, the sector holding the FAT chain tail is rewritten (every 4-8KB of growth at 4-8KB clusters), so FAT1+FAT2 metadata writes are sprinkled continuously through the body, and a torn FAT sector is shared - it can crosslink and destroy many files and directories at once.

raptor#46 makes rmr record under <name>.mp4.tmp and publish via rename() after truncate + fsync, plus opt-in recording.prealloc_mb (ftruncate() reservation, metadata-only on vfat/exfat): the FAT chain is built once at open and freed once at close; in between, only data clusters are written.

Honest status: tests written but not executed, no device validation yet. The two PRs are independent layers - how dirty data reaches the card, vs. what filesystem metadata a crash can catch mid-write.

[nicolasroche898]

This is relevant also to NFS mount. right now on slow network the whole 20% memory will be used as dirty page cache, potentially causing memory starvation on other services. this change limits the dirty page cache to 4MB, about 10% of memory.

Alternatively, if we're ok with 20% dirty page cache for NFS, as it is currently defined, then we should be ok with (same) 20% for sdcard too. Either way, its good to have this number known and purposely chosen to fit thingino system requirements

[WLTB-Gino]

Following up on @nicolasroche898's question 3 (ratio vs bytes): swept every camera defconfig against the SoC RAM maps to answer "is any supported board's stock 20% ceiling already at or below the pinned 4MB?"

No. The stock ratio ceiling bottoms out above 4MB on every supported board:

• Aobocam A12 (T23DL, the only 32MB board in the tree): 22MB Linux -> ~4.4MB nominal max dirty. Closest to the pin, still above it.
• 64MB boards: Jooan A6M (T23N) ~8.6MB; tightest 3MP T31L recorders (aosu C5L 31MB, dekco DC5L / LaView L2 32MB) ~6.2-6.4MB; T20L/T10L ~10MB.
• 128MB boards: >=11MB (e.g. Vanhua Z55/Z55i: 64MB Linux -> ~12.8MB); 256MB SoCs ~38MB.

(Caveat: the 20% is of dirtyable memory, not total Linux RAM, so effective ceilings on a loaded camera run lower and float during runtime — that non-determinism is exactly what the byte pin removes.)

So the 4M hard cap is at-or-below stock behavior fleet-wide: a genuine tightening on almost every board, near-neutral only on the A12.