SER silent-failure guards: Phases 1-7 (loud-fail guard + encoder registry + problem_type + Phase 2 warning + diagnostics CLI)

[satra]

Summary

Implements all phases of specs/20260509-204500-ser-silent-failure-guards/plan.md. Targets PR #511's branch (fix/ser-wav2vec2-regression-head) so it stacks on the in-flight work.

Two reviewer passes: a quick "ship it" pass that landed Phases 1–7, then a strict adversarial pass against the user's "zero tolerance for silent failures" criterion that found 5 critical defects and forced changes to the routing/guard semantics. The current state addresses all 5.

Commits

1. 1ce3453 Phase 1 — loud-fail post-load assertion (missing_keys + mismatched_keys + final-layer out_features shape sanity).
2. 11d5800 Phases 2–7 — registry refactor, problem_type detection, standard-pipeline warning, doc + CLI.
3. 9b92b72 First-pass reviewer cleanups — explicit encoder class names in _BASE_REGISTRY, CLI null-config note.
4. 0849f3d mypy --extra-checks annotation fix from CI.
5. bb4de3b Strict reviewer second pass — 5 silent-failure surfaces closed:
   • D1: dropped wav2vec2-bert from _BASE_REGISTRY. It uses log-mel input_features via SeamlessM4TFeatureExtractor, not raw input_values. Re-introducing requires a per-family preprocessing adapter.
   • D2: Phase-1 guard now treats both classifier.* and {encoder_attr}.* missing/mismatched keys as suspect (with a small whitelist for HF post_init buffers). A wrong _BASE_REGISTRY entry now fails loudly instead of silently scrambling encoder weights.
   • D3: _resolve_apply_softmax now peeks architectures when problem_type is absent. Any model declaring Wav2Vec2ForSpeechClassification (audeering signature) is treated as regression — closes the silent-corruption window where a regression checkpoint with English-emotion labels would have been softmaxed.
   • D4: _peek_head_final_layer now raises when classifier.dense.weight is present but no recognized final-layer attribute matches, instead of returning None and letting the standard pipeline silently random-init the head. Error message lists the actual classifier keys to guide adding to _FINAL_LAYER_NAMES/_KNOWN_HEAD_LAYOUTS.
   • D5: SENSELAB_STRICT_HEAD_LOAD default inverted to =1 (raise). Warn-only let downstream consumers like windowed segmentation aggregate corrupted scores. Set SENSELAB_STRICT_HEAD_LOAD=0 to demote to warning for intentional partial loads (fine-tuning warm-start).
   • T203: 3 unit tests for _check_head_loaded_cleanly cover strict-by-default raise, warn-only mode, and silent pass on clean loads.

What changed (by phase)

Phase 1 — Custom-path loud-fail (api.py)

After EmotionModel.from_pretrained(..., output_loading_info=True):

• Raise on any suspect missing/mismatched key under classifier. or {encoder_attr}. (the Phase-1 guard updated in commit bb4de3b per D2).
• Raise if final layer's out_features ≠ config.num_labels.

Phase 2 — Standard-pipeline guard (huggingface.py)

• _get_hf_audio_classification_pipeline constructs AutoModelForAudioClassification.from_pretrained(..., output_loading_info=True) and threads the loaded instance into pipeline(model=..., feature_extractor=...). No double download.
• _check_head_loaded_cleanly raises by default on suspect head keys (classifier., head., score., out_proj., projector.).
• SENSELAB_STRICT_HEAD_LOAD=0 demotes to a warning.

Phase 3 — Encoder-family generalization (api.py)

• _BASE_REGISTRY: dict[str, tuple[str, str, str]] covers wav2vec2/hubert/wavlm (wav2vec2-bert dropped per D1). Class names explicit; resolved lazily.
• _make_emotion_model_class(model_type, head) uses the right base class + encoder attribute name.
• _emotion_head_kind returns (model_type, _HeadEntry).

Phase 4 — Head registry (api.py)

• _HeadEntry dataclass: (final_layer, activation, dropout_field).
• _KNOWN_HEAD_LAYOUTS: dict[str, _HeadEntry] — currently the ehcalabres single-bin entry.
• _make_regression_head_class(head) builds the matching head; _ACTIVATIONS table covers tanh/gelu/relu.

Phase 5 — problem_type detection (api.py)

• _resolve_apply_softmax(model, ser_type) prefers config.problem_type; falls back to architecture peek (ForSpeechClassification → regression) per D3; finally to keyword heuristic.

Phase 6 — Resampling (no code change)

Verified classify_audios_with_transformers already resampled in its audio loop.

Phase 7 — Docs + diagnostic CLI

• doc.md enumerates dispatch paths and points users at the CLI.
• python -m senselab.audio.tasks.classification.speech_emotion_recognition --probe <repo_id> reports model_type, architectures, dispatch decision, head source, and emits advisories. No weights loaded.

Caveats this PR explicitly does NOT solve

• wav2vec2-bert intentionally not registered (see D1).
• Encoder forward signature drift for batched inference (T305): single-clip path is verified for wav2vec2/hubert/wavlm; batched HuBERT/WavLM may need attention_mask threading. Phase-1 guard catches missing weights, not forward-signature mismatches.
• Multi-task heads (assumption switching to senselab and pydra #9): a checkpoint with both regression and classification branches under one classifier whose problem_type covers only one branch is not handled.
• Custom preprocessor configs (assumption Add speaker diarization #11): we use Wav2Vec2FeatureExtractor.from_pretrained defaults; do_normalize/return_attention_mask/pre-emphasis overrides in preprocessor_config.json are silently ignored.

Backward compatibility

Legacy aliases retained as 2-line trampolines so external monkeypatching keeps working:

• _make_wav2vec2_emotion_model_class(final_layer)
• _wav2vec2_emotion_head_kind(model) returning just the final-layer name
• _peek_wav2vec2_head_final_layer(model)
• _classify_wav2vec2_speech_cls_ser(..., final_layer="out_proj")

Test plan

• uv run pytest src/tests/audio/tasks/classification_test.py -q — all pass (1 GPU-skipped).
  • test_speech_emotion_recognition_continuous (audeering happy path).
  • test_wav2vec2_speech_cls_ser_raises_on_random_head (parametrised: missing + mismatched).
  • test_wav2vec2_speech_cls_ser_raises_on_shape_mismatch (final-layer out_features sanity).
  • test_check_head_loaded_cleanly_raises_strict_by_default (parametrised across head prefixes).
  • test_check_head_loaded_cleanly_warns_when_lax (env-var demotion).
  • test_check_head_loaded_cleanly_silent_on_clean_load (encoder-buffer whitelist).
• uv run ruff check + uv run ruff format --check clean.
• uv run mypy --ignore-missing-imports --extra-checks clean.
• CLI smoke-tested against audeering, ehcalabres, superb.

https://claude.ai/code/session_01DyXMQNsF7F5Ze8j3GGVw66

[gemini-code-assist]

Code Review

This pull request implements Phase 1 of a plan to eliminate silent-failure modes in speech emotion recognition by adding a plan document and implementing post-load assertions for classifier keys and output shapes. New unit tests verify these safeguards. Review feedback suggests optimizing a list comprehension with the walrus operator and updating the plan document to mark the unit testing task as completed.