ScriptLine: accept empty text as "model produced no transcript" signal

[wilke0818]

Summary

Stop rejecting ScriptLine(text="") as invalid when its speaker field is also unset. Empty text is a valid "the ASR was called but produced no transcript" signal — distinct from text=None ("the ASR was never called"). The pre-change validator collapsed both into a single failure case, which caused legitimately-empty ASR output to crash the audio-analysis pipeline.

Why

Surfaced during a full-pipeline run on a b2ai pediatric voice subject. Out of 92 audio clips, 5 failed with:

[asr[ibm-granite/granite-speech-3.3-8b]] FAILED in 0.1s:
  1 validation error for ScriptLine
  Value error, At least text or speaker must be provided.
  [type=value_error, input_value={'text': ''}, input_type=dict]

All 5 were the same shape: short clips (1.4–3.4 s) on the enhanced_16k pass where Granite-Speech correctly decoded "no recognizable speech" and emitted {"text": ""}. The ScriptLine validator then rejected the construction, the per-clip ASR result was lost, and the pass's PII / alignment / corroboration logic ran with one ASR backend silently missing.

The vulnerability is shared across every senselab ASR backend (nemo.py:145, granite.py:142, canary_qwen.py:191, qwen.py:223, huggingface.py:284 via .from_dict()) — each one constructs ScriptLine(text=...) directly. Granite-Speech happens to be the one that triggers it in this dataset because it's the most honest of the bunch: when its acoustic model decodes nothing, it returns "". Whisper hallucinates filler ("Thanks for watching!", "..."), and the LLM-augmented Canary-Qwen and Qwen3-ASR's instruct-tuned heads almost always emit something under their "Transcribe the following:" prompt, so they slip past the same validator. The bug is in ScriptLine, not Granite.

The fix

src/senselab/utils/data_structures/script_line.py:78-93 — single-line semantic change from a falsiness check to a presence check:

# Before
if not values.get("text") and not values.get("speaker"):
    raise ValueError("At least text or speaker must be provided.")

# After
if values.get("text") is None and values.get("speaker") is None:
    raise ValueError("At least text or speaker must be provided.")

In Python not "" is True. The previous rule rejected text="" exactly the same as text=None or missing-entirely. The new rule treats text="" and text=" " and text="hello" as all "provided" — only None (or absent) counts as "missing." Whitespace-only strings remain valid: the existing field validator strips them, so text=" " and text="" converge on the same final state. (Previously they took different validator paths and text=" " survived while text="" did not — a real inconsistency this also closes.)

Behavior preserved unchanged:

• ScriptLine(text="hello") — works (it always did).
• ScriptLine(speaker="spk1") — works (diarization-only line).
• ScriptLine() with both fields absent or both None — still raises.
• Negative-timestamp validation, whitespace-stripping, all other field validators — untouched.

Behavior change

• ScriptLine(text="") now constructs cleanly (was raising ValidationError).
• ScriptLine(text="", start=0.0, end=1.5) now constructs cleanly.
• ScriptLine.from_dict({"text": ""}) now constructs cleanly.

Downstream consumers (global_summary, the harvesters, the cross-ASR corroboration in pii.py) all already handle empty / whitespace text gracefully — they filter it from _build_full_text, they check if t.strip() before scanning, they drop empty utterances from voting. So letting an empty ScriptLine exist in the report just means the "Granite was called on this clip and decided there's no speech" signal is preserved rather than discarded. No downstream code needs updating.

Tests

New file src/tests/utils/data_structures/script_line_test.py — 11 tests covering both directions of the rule.

Empty / whitespace / None text handling:

• test_empty_text_without_speaker_is_accepted — ScriptLine(text="") now constructs.
• test_whitespace_only_text_is_accepted_and_normalized_to_empty — text=" " survives validation, gets stripped to "". Fixes the inconsistency where " " was previously accepted but "" wasn't.
• test_text_none_without_speaker_still_rejected — ScriptLine(), ScriptLine(text=None), and ScriptLine(text=None, speaker=None) all still raise.

Speaker-only / mixed:

• test_only_speaker_provided_is_accepted — ScriptLine(speaker="spk1") works (diarization line with no transcript).
• test_empty_text_with_speaker_is_accepted — ScriptLine(text="", speaker="spk1") works (speaker turn detected, ASR returned nothing for it — common at silence boundaries).

Happy-path regression:

• test_normal_text_unchanged — ScriptLine(text="hello world") constructs and preserves the text.
• test_text_stripped_of_surrounding_whitespace — text=" hello " strips to "hello". Unchanged behavior; asserted to guard against future regressions.

from_dict:

• test_from_dict_accepts_empty_text_entry — ScriptLine.from_dict({"text": ""}) now constructs.
• test_from_dict_rejects_dict_with_no_text_or_speaker — ScriptLine.from_dict({}) still raises.

Field-validator interaction:

• test_empty_text_with_timestamps_is_accepted — ScriptLine(text="", start=0.0, end=1.5) works.
• test_negative_timestamp_still_rejected_independent_of_text — timestamp validator stays orthogonal; ScriptLine(text="", start=-1.0) still raises.

All 11 pass under --noconftest. Pre-commit (ruff + mypy + codespell) clean.

The existing forced_alignment_test.py (the only other test that constructs ScriptLine via .from_dict) continues to pass — it uses a real fixture, not an empty-text edge case.

Test plan

• uv run pytest src/tests/utils/data_structures/script_line_test.py --noconftest — 11 / 11 pass.
• Verified the same 5 Granite-Speech failures from the original b2ai run no longer manifest: re-running the smoke path with Granite.transcribe_with_granite_speech(audio) returning [ScriptLine(text="")] from the worker no longer raises at the host-side construction site.
• Out-of-band: re-run bash scripts/submit_subject_audio.sh /path/to/dataset sub-XXX on a subject that previously had Granite failures and confirm exit code 0 across all clips, with the affected clips' summary.json now showing Granite-Speech as ok rather than dropped.

Out-of-band reviewer notes

Why this is a ScriptLine change, not a granite.py change. Could fix this in the Granite wrapper alone (e.g., granite.py:142 filters empty-text and returns [] instead of [ScriptLine(text="")]). That would silence the symptom for Granite but leave the latent vulnerability in every other backend — the moment a future Whisper / Canary / Qwen build returns an empty transcript the bug resurfaces. Fixing it once in the validator addresses the class of bug.

Why empty is a meaningful signal. "ASR was called, returned nothing" is genuinely different from "ASR was never called." The first is evidence that the audio doesn't contain decodable speech according to this model; multiple ASRs returning empty on the same clip is a corroborated "no-speech" signal. Collapsing both into text=None / construction-failure throws away that distinction. The pipeline's existing cross-ASR-model corroboration logic (used by pii.py and indirectly by global_summary) is happier with empty ScriptLines than with missing ones.

No call-site changes needed. Every ScriptLine(text=...) construction site I traced (5 in senselab/audio/tasks/speech_to_text/, plus talk_bank_helpers.py) was already passing along whatever the upstream model returned. The change just allows the empty case through; it doesn't require existing producers to do anything new.

Branch is independent of feat/pii-subprocess-venv — they touch unrelated files and either can merge first. They're filed off alpha separately to keep the diffs and blame surfaces clean.

Version

Published prerelease version: 1.3.1-alpha.33

Changelog

🐛 Bug Fix

• ScriptLine: accept empty text as "model produced no transcript" signal #520 (@wilke0818)
• Feat/pii subprocess venv #519 (@wilke0818)
• subprocess_venv: route torch/torchaudio via CUDA-aware PyTorch index #516 (@satra @wilke0818)
• subprocess_venv: two-stage install to fix torch/torchaudio CUDA-tag s… #518 (@wilke0818)
• Fix/ser wav2vec2 regression head #511 (@satra @wilke0818 @claude)
• comparison & uncertainty stage for analyze_audio.py #512 (@satra)
• audio analysis script + multilingual MMS aligner + 4 new ASR backends #510 (@satra)

Authors: 3

• Claude (@claude)
• Jordan Wilke (@wilke0818)
• Satrajit Ghosh (@satra)

[gemini-code-assist]

Code Review

This pull request updates the ScriptLine data structure to allow empty strings for text and speaker fields, distinguishing them from None to represent valid empty transcripts. Comprehensive tests were added to verify this behavior. Feedback highlights an inconsistency in the from_dict method where an internal helper still uses truthiness checks, which would cause empty strings to be filtered out during deserialization. It is recommended to update this helper and add tests for nested chunks to ensure consistent behavior.