Commit 38f86b2
feat: add preflight validation workflow and centralize shared validators (#406)
`tooling/` is where the evaluation-report renderer will eventually
consolidate and where agent-friendly JSON and plain-text modes will
land. Keeping preflight rendering-free means render swaps never reach
into check code.
## UX
`safe-synthesizer run --validate ...` prints a Rich report grouped by
stage, with per-check status, severity-tagged issues, and a terminal
summary. Errors set exit code 1. Warnings are advisory. See
`docs/user-guide/running.md` and `docs/user-guide/troubleshooting.md`.
## Extending preflight
`docs/developer-guide/preflight-plugins.md` walks two end-to-end worked
examples (`DuplicateRowsCheck`, `GroupSkewCheck`), covers the `requires`
/ `enabled()` dependency model and issue-code conventions, and includes
a "Testing a check" section.
## Tests
- `tests/preflight/test_preflight.py`: per-check unit coverage.
- `tests/preflight/test_plugin_registration.py`: plugin surface,
reserved namespaces, crash isolation, dependency gating.
- `tests/preflight/test_registry_validation.py`: `build_registry` and
`_validate_registry` shape invariants.
- `tests/preflight/test_preflight_cli.py`: smoke coverage for the
`--validate` path.
- `tests/cli/test_run.py`: integration with `safe-synthesizer run`.
## Follow-ups
- `safe-synthesizer doctor`: a data-independent environment health
command that reuses the environment and config-only subset of the
registry. Plan drafted, implementation deferred.
- Plain, JSON, and agent-friendly render modes via
`tooling.modes.RenderMode`.
closes #332.
---------
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Kendrick Boyd <kendrickb@nvidia.com>
Signed-off-by: aaron gonzales <aagonzales@nvidia.com>
Signed-off-by: aagonzales <aagonzales@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Kendrick Boyd <kendrickb@nvidia.com>1 parent bf4754e commit 38f86b2
64 files changed
Lines changed: 5853 additions & 297 deletions
File tree
- docs
- developer-guide
- user-guide
- src/nemo_safe_synthesizer
- cli
- config
- data_processing
- holdout
- llm
- pii_replacer
- preflight
- checks
- sdk
- tooling
- _rich
- training
- tests
- cli
- config
- data_processing
- llm
- observability
- preflight
- sdk
- training
- tools/codestyle
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | 76 | | |
80 | 77 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
59 | | - | |
| 59 | + | |
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
65 | 67 | | |
66 | 68 | | |
67 | 69 | | |
| |||
0 commit comments