Skip to content

Commit 1b3d93c

Browse files
authored
docs: removed redundant files/code and added docstrings for artifacts/ (#175)
## Summary - Deleted 15 dead files: - entire DataChecksProcessor pipeline - data_checks_processor.py - base/manifest.py - base/metadata.py - base/metrics.py - base/analyzer.py - base/name_anonymizer.py - base/data_checks.py - Entire analyzers/data_checks/ directory (8 files): - __init__.py - base.py - dataset_size.py - high_float_precision.py - missing_data.py - sparse_data.py - surrounding_whitespaces.py - text_data.py - Partial cleanup in 3 files: - `analyzers/field_features.py`: Removed `FieldFeaturesAnalyzer` class, 5 dead helper functions (`_add_info_to_manifest`, `_add_ner_to_fields`, `_type_summary`, `_calculate_aggregates`, `_max_aggregate`), and 8 dead imports. - `base/fields.py`: Removed `FieldFeaturesInfo` class and unused `cached_property` import. - `base/__init__.py`: Removed unused `ArtifactProcessingException`. - Added docstrings to all 5 remaining files: Ensure that the following pass: - [x] `make format && make check` or via prek validation. - [x] `make test` passes locally - [x] `make test-e2e` passes locally - [ ] `make test-ci-container` passes locally (recommended) ## Other Notes made by Cursor Signed-off-by: Sean Yang <seayang@nvidia.com>
1 parent 334bd1c commit 1b3d93c

17 files changed

Lines changed: 109 additions & 1363 deletions

src/nemo_safe_synthesizer/artifacts/analyzers/data_checks/__init__.py

Lines changed: 0 additions & 36 deletions
This file was deleted.

src/nemo_safe_synthesizer/artifacts/analyzers/data_checks/base.py

Lines changed: 0 additions & 131 deletions
This file was deleted.

src/nemo_safe_synthesizer/artifacts/analyzers/data_checks/dataset_size.py

Lines changed: 0 additions & 39 deletions
This file was deleted.

src/nemo_safe_synthesizer/artifacts/analyzers/data_checks/high_float_precision.py

Lines changed: 0 additions & 71 deletions
This file was deleted.

src/nemo_safe_synthesizer/artifacts/analyzers/data_checks/missing_data.py

Lines changed: 0 additions & 64 deletions
This file was deleted.

0 commit comments

Comments
 (0)