[MAINTENANCE] Add SQL result_format regression coverage for expect_column_values_to_be_unique - #11964
Closed
joshua-stauffer wants to merge 9 commits into
Closed
[MAINTENANCE] Add SQL result_format regression coverage for expect_column_values_to_be_unique#11964joshua-stauffer wants to merge 9 commits into
joshua-stauffer wants to merge 9 commits into
Conversation
…_be_unique Pin down behaviors of the column_values.unique metric on SQL backends that are easy to break when its implementation changes: - COMPLETE + unexpected_index_column_names composed with a row_condition - include_unexpected_rows composed with a row_condition - unexpected_index_query must execute against the source database and return exactly the unexpected rows - exclude_unexpected_values must return the columnar index-list shape shared by all SQL map expectations - importing great_expectations must not emit metric re-registration warnings on stderr
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
joshua-stauffer
marked this pull request as ready for review
July 10, 2026 09:23
- Narrow batch.datasource to SQLDatasource before calling get_engine() (mypy attr-defined) - Allow 30s for the subprocess import test; the default unit timeout of 2s is too tight for a full library import
The ignore on the NotImported fallback assignment is only needed when pyarrow is installed (mypy then types the name as Module). In environments without pyarrow the module resolves as Any, and --warn-unused-ignores fails the build. Adding unused-ignore makes the ignore valid in both environments.
…thub.com/fivetran/great_expectations into m/unique-result-format-regression-guards
joshua-stauffer
marked this pull request as draft
July 22, 2026 08:34
leodrivera
added a commit
to leodrivera/great_expectations
that referenced
this pull request
Jul 22, 2026
…nique - Fix KeyError when row_condition is present: _named_source_subquery now always wraps the domain selectable in an explicit named projection - Register a dedicated unexpected_index_query provider using the same join-back pattern as the other row-retrieval paths, so the query string surfaced in validation results and Data Docs is executable against the source database - Reuse _get_sqlalchemy_customized_unexpected_index_list so exclude_unexpected_values returns the canonical columnar index-list shape - Pull result_format regression tests from fivetran#11964 The metric re-registration warnings on import remain; resolving them depends on a MapMetricProvider rework to support custom auxiliary providers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
these tests will merge via #11863 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds regression coverage for
ExpectColumnValuesToBeUniqueresult_format behaviors on SQL backends (PostgreSQL, SQLite). These behaviors depend on internals of thecolumn_values.uniquemetric that are easy to break when its SQL implementation changes, and were previously untested:COMPLETE+unexpected_index_column_namescomposed with arow_condition— the test data is constructed so a value is duplicated only when the filtered-out row is included, proving the condition is applied and the index list hydrates correctly.include_unexpected_rowscomposed with arow_condition.unexpected_index_querymust be executable against the source database and return exactly the unexpected rows (the query string is surfaced to users in validation results and Data Docs).exclude_unexpected_values=Truemust return the columnar{index_column: [values, ...]}index-list shape shared by all SQL map expectations.great_expectationsmust not emit metric re-registration warnings on stderr.Changes
tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_be_unique.py: five new tests plus a dedicated dataset and a small helper that fails loudly if any metric raised instead of producing a value.Testing
develop.