Skip to content

fix: tune segmentation to match training and correct the engine size - #29

Merged
Endika merged 1 commit into
mainfrom
fix/segmenter-and-size
Aug 5, 2026
Merged

fix: tune segmentation to match training and correct the engine size#29
Endika merged 1 commit into
mainfrom
fix/segmenter-and-size

Conversation

@Endika

@Endika Endika commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Two things, both found by testing the deployed app rather than the model.

The app was feeding the model worse input than it trained on

Training resamples every frame of a recording; the app resamples whatever SignSegmenter emits. Nobody was measuring the second one. tools/train/simulate_app.py replays the real test split through a port of the segmenter and scores the real weights:

whole recording (what training sees)   top1 0.741  top3 0.870
app segmenter, before                  top1 0.722  top3 0.848
app segmenter, after                   top1 0.739  top3 0.868

Two constants: motion threshold 0.08 → 0.03, settle frames 6 → 10. Swept over the test set rather than guessed. The lower threshold also fixes gentle signing, which at 0.08 never crossed the activation line at all — the segmenter stayed idle and the vocabulary engine was never asked.

A correction to my own earlier finding. I first measured this gap at 12 points and called it a deployed bug. It was my simulation: it read only the right hand, so every left-handed sign registered zero motion, never activated, and was dropped. 49 of 598 signs, all of them an artifact of the measurement. The real gap was 1.9 points. Recording it because the wrong number was alarming and the right one is not.

The engine size on screen was wrong

The panel said 41 MB. That is what sits in the repo, and it includes both WASM builds — SIMD and no-SIMD — of which any given browser downloads exactly one:

models 19.0 MB
WASM, SIMD 11.5 MB
WASM, no-SIMD 10.8 MB (not downloaded)
real download 30.5 MB

Reported as ~30 MB now.

And a real gap it exposed

Reported from the app: after pressing "descargar ahora" the panel showed 19 MB — exactly the models, with the WASM missing. Weights cached without the runtime that executes them looks fine online and breaks offline, which is a front-page claim.

I cannot reproduce this without a browser, so instead of guessing at a fix I made it visible: report() now returns hasRuntime, and the panel says so outright when the runtime is absent. If it still shows 19 MB and that warning after this ships, the service worker is not intercepting MediaPipe's WASM fetch and that is the next thing to chase.

@Endika
Endika merged commit 57a0949 into main Aug 5, 2026
7 checks passed
@Endika
Endika deleted the fix/segmenter-and-size branch August 5, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant