Skip to content

fix: recognise signing that never pauses - #25

Merged
Endika merged 1 commit into
mainfrom
fix/continuous-signing
Aug 5, 2026
Merged

fix: recognise signing that never pauses#25
Endika merged 1 commit into
mainfrom
fix/continuous-signing

Conversation

@Endika

@Endika Endika commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Reported from testing against a YouTube video of someone signing: the app detected nothing at all.

The bug

SignSegmenter closed a window only when the hand held still for six consecutive frames. That is how isolated dictionary signs are delimited, and it is how SWL-LSE was recorded — but a fluent signer never holds still.

Measured on 300 frames of continuous motion:

before:  300 frames -> 0 windows closed
after:   300 frames -> 6 windows closed

Zero. The vocabulary engine was never invoked once, so the app could not have recognised anything no matter how good the model was. It looked broken rather than inaccurate, which is the worse failure of the two.

The fix is a hard cap: if the window reaches maxFrames while motion continues, emit it anyway. Stillness is still preferred when it happens, so isolated signs are not chopped mid-sign — there is a test for that.

Found a second thing while probing

My first probe reported 0 windows after the fix too. The synthetic motion I wrote came out at 0.077 palm-widths per frame against a 0.08 threshold — it was never activating the segmenter, so I was measuring nothing. Worth recording because it applies to real use as well: gentle or small-amplitude signing sits under the activation threshold and will not trigger the vocabulary engine. That is tuning, and tuning needs real footage rather than synthetic hands, so it is not changed here.

What this does not fix

Being honest about the limits, because this change will not make that video work:

  1. The model is trained on isolated signs. Each SWL-LSE recording is one sign performed alone, with lead-in and lead-out. Continuous signing co-articulates — signs blend into their neighbours. Accuracy on fluent signing will be far below the 74% measured on isolated test samples.
  2. The vocabulary is 238 clinical concepts. A random signing video is unlikely to be discussing acúfenos or colonoscopias, and MIN_CONFIDENCE correctly stays silent rather than guessing.

So: the app will now try on continuous signing, where before it structurally could not. It will not read a fluent conversation.

@Endika
Endika merged commit 1591ebe into main Aug 5, 2026
7 checks passed
@Endika
Endika deleted the fix/continuous-signing branch August 5, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant