Commit 1b3d93c
authored
docs: removed redundant files/code and added docstrings for artifacts/ (#175)
## Summary
- Deleted 15 dead files:
- entire DataChecksProcessor pipeline
- data_checks_processor.py
- base/manifest.py
- base/metadata.py
- base/metrics.py
- base/analyzer.py
- base/name_anonymizer.py
- base/data_checks.py
- Entire analyzers/data_checks/ directory (8 files):
- __init__.py
- base.py
- dataset_size.py
- high_float_precision.py
- missing_data.py
- sparse_data.py
- surrounding_whitespaces.py
- text_data.py
- Partial cleanup in 3 files:
- `analyzers/field_features.py`: Removed `FieldFeaturesAnalyzer` class,
5 dead helper functions (`_add_info_to_manifest`, `_add_ner_to_fields`,
`_type_summary`, `_calculate_aggregates`, `_max_aggregate`), and 8 dead
imports.
- `base/fields.py`: Removed `FieldFeaturesInfo` class and unused
`cached_property` import.
- `base/__init__.py`: Removed unused `ArtifactProcessingException`.
- Added docstrings to all 5 remaining files:
Ensure that the following pass:
- [x] `make format && make check` or via prek validation.
- [x] `make test` passes locally
- [x] `make test-e2e` passes locally
- [ ] `make test-ci-container` passes locally (recommended)
## Other Notes
made by Cursor
Signed-off-by: Sean Yang <seayang@nvidia.com>1 parent 334bd1c commit 1b3d93c
17 files changed
Lines changed: 109 additions & 1363 deletions
File tree
- src/nemo_safe_synthesizer/artifacts
- analyzers
- data_checks
- base
Lines changed: 0 additions & 36 deletions
This file was deleted.
Lines changed: 0 additions & 131 deletions
This file was deleted.
Lines changed: 0 additions & 39 deletions
This file was deleted.
Lines changed: 0 additions & 71 deletions
This file was deleted.
Lines changed: 0 additions & 64 deletions
This file was deleted.
0 commit comments