Skip to content

Enabling guardrails silently skips video content safety: classifier is commented out and the check returns safe with no warning #170

Description

@stewtong

Update 2026-08-09: option 1 implemented in #171.

Running generation with guardrails enabled at commit 5e67049cd94acb667786f1e6dd0dab821cb90c97 applies the text guardrail (Blocklist + Qwen3Guard) and the RetinaFaceFilter face blur, but performs no video content-safety screening. VideoContentSafetyFilter is commented out in cosmos_framework/auxiliary/guardrail/common/presets.py (lines 27-28 at that commit) with the note "Too many false positives, add back when fixed", and at runtime the check logs:

No safety models found, returning safe

The generation completes normally and reports guardrails as enabled.

Observed serving nvidia/Cosmos3-Super (text2video and image2video, 1280x720, 189 frames, 35 steps, guardrails enabled) on 8x H200. Every output passed the video safety stage by construction, because the stage runs no model.

Why this matters: an operator who enables guardrails has good reason to believe video content screening is active. The log line reads as a successful check with a safe verdict rather than as a skipped check, and downstream tooling that records the guardrail setting will faithfully report guardrails as enabled while the effective posture is text screening plus face blur only. For teams using the guardrail stack in safety-relevant or compliance-relevant pipelines, the gap between the configured posture and the effective posture is silent.

Suggested fix, in increasing order of effort:

  1. Log an explicit warning when guardrails are enabled but the video content-safety stage has no model to run, e.g. WARNING: video content safety classifier is disabled at this release; effective guardrail coverage is text checks and face blur only. A warning distinguishes "checked and safe" from "not checked".
  2. Document the effective guardrail coverage per release in the guardrail section of the docs, so the commented-out state is discoverable without reading presets.py.
  3. When the classifier returns, gate its re-enablement behind a config flag so the posture is explicit in either state.

Happy to open a PR for option 1 if that is useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions