[MAINTENANCE] Remove dead statistical expectations and schemas#11764
[MAINTENANCE] Remove dead statistical expectations and schemas#11764joshua-stauffer wants to merge 3 commits into
Conversation
Removes dead statistical expectations (bootstrapped KS, chi-square, Cramers phi, parameterized distribution KS), their metric, and the unused core/schemas/ directory. Cleans up string references in self_check/util.py.
✅ Deploy Preview for niobium-lead-7998 canceled.
|
There was a problem hiding this comment.
Pull request overview
Removes expectation implementations and JSON schema artifacts identified as unreachable from @public_api roots, reducing dead code and package surface area.
Changes:
- Deleted multiple incomplete / unexported statistical Expectation classes.
- Removed unused JSON schema definitions under
great_expectations/expectations/core/schemas/.
Reviewed changes
Copilot reviewed 45 out of 59 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| great_expectations/expectations/core/schemas/ExpectQueryResultsToMatchComparison.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectCompoundColumnsToBeUnique.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToNotMatchRegexList.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToNotMatchRegex.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToNotMatchLikePatternList.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToNotMatchLikePattern.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToNotBeNull.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchRegex.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchLikePattern.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToBeUnique.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToBeNull.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValuesToBeInTypeList.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnValueLengthsToEqual.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnToExist.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnSumToBeBetween.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnStdevToBeBetween.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnPairValuesToBeEqual.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnMinToBeBetween.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/schemas/ExpectColumnMedianToBeBetween.json | Removes unused expectation JSON schema. |
| great_expectations/expectations/core/expect_column_parameterized_distribution_ks_test_p_value_to_be_greater_than.py | Removes incomplete / unreachable expectation class. |
| great_expectations/expectations/core/expect_column_pair_cramers_phi_value_to_be_less_than.py | Removes incomplete / unreachable expectation class and renderers. |
| great_expectations/expectations/core/expect_column_chisquare_test_p_value_to_be_greater_than.py | Removes incomplete / unreachable expectation class. |
| great_expectations/expectations/core/expect_column_bootstrapped_ks_test_p_value_to_be_greater_than.py | Removes incomplete / unreachable expectation class. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #11764 +/- ##
===========================================
+ Coverage 84.59% 84.82% +0.23%
===========================================
Files 473 468 -5
Lines 39260 39135 -125
===========================================
- Hits 33211 33198 -13
+ Misses 6049 5937 -112 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
NathanFarmer
left a comment
NathanFarmer
left a comment
There was a problem hiding this comment.
These schemas are more of a DX tool for viewing schema changes in PRs
The schemas directory alerts developers to schema changes created by updating Expectations (via 'invoke schemas --sync'). It was incorrectly flagged as dead code because nothing imports it at runtime.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "expect_column_values_to_be_dateutil_parseable", | ||
| "expect_column_values_to_be_json_parseable", | ||
| "expect_column_values_to_match_json_schema", | ||
| "expect_column_stdev_to_be_between", | ||
| "expect_column_most_common_value_to_be_in_set", | ||
| "expect_column_bootstrapped_ks_test_p_value_to_be_greater_than", | ||
| "expect_column_parameterized_distribution_ks_test_p_value_to_be_greater_than", | ||
| "expect_column_pair_values_to_be_equal", | ||
| "expect_column_pair_values_a_to_be_greater_than_b", |
There was a problem hiding this comment.
This PR removes several expectation types from the codebase, but there are still references to these expectation type strings elsewhere (e.g., docs/docusaurus/versioned_docs/version-0.18/reference/learn/expectations/distributional_expectations.md, assets/scripts/build_gallery.py, and tests/expectations/test_expectation_atomic_renderers.py). Also, the PR description states the entire great_expectations/expectations/core/schemas/ directory was removed, but it still exists in-tree. Please either update/remove those remaining references (and delete the schemas dir if intended), or adjust the PR description/scope to match what’s actually changing.
|
Is this PR still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions 🙇 |
github-actions
Bot
added
stale
|
Is this PR still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions 🙇 |
Summary
Removes dead code identified by the dead code analyzer (AST-based reachability analysis from
@public_apiroots).Removed
great_expectations/expectations/core/expect_column_bootstrapped_ks_test_p_value_to_be_greater_than.pygreat_expectations/expectations/core/expect_column_chisquare_test_p_value_to_be_greater_than.pygreat_expectations/expectations/core/expect_column_pair_cramers_phi_value_to_be_less_than.pygreat_expectations/expectations/core/expect_column_parameterized_distribution_ks_test_p_value_to_be_greater_than.pygreat_expectations/expectations/metrics/column_aggregate_metrics/column_bootstrapped_ks_test_p_value.pygreat_expectations/expectations/core/schemas/(entire directory — unused JSON schemas)self_check/util.pyVerification performed
@public_apidecorators on removed codecloudorgx-runner__init__.pyre-export references remainingdead_code_exceptions.jsonGenerated with Claude Code using the dead-code-removal skill.