Skip to content

[MAINTENANCE] Type-check tests/expectations/ #12131

Description

@joshua-stauffer

Part of #12128.

What to do

One exclude pattern, tests/expectations, hides the whole directory — 50 modules, of which
16 carry a total of 134 errors. Fix them, then remove that single pattern and regenerate the
relaxation inventory in the same change. Because the pattern is directory-level, every file
under tests/expectations/ has to check clean before it can come out; replacing it with a
narrower pattern is not an option.

Fix each error by correcting the test against the library's current contract, never by
silencing it.

Where

Errors by file — the other 34 modules already check clean:

  • metrics/test_metrics_util.py (28), test_expectation_atomic_renderers.py (25),
    core/test_expect_column_values_to_not_be_outliers.py (20),
    metrics/query_metrics/test_query_metrics.py (18),
    core/test_unexpected_rows_expectation.py (8), metrics/test_metric_providers.py (8),
    metrics/column_map_metrics/test_column_values_in_set.py (6),
    fixtures/expect_column_values_to_equal_three.py (4), test_expectation_diagnostics.py (4),
    metrics/conftest.py (3), test_expectation.py (3), metrics/test_core.py (2),
    metrics/test_map_metric.py (2), and one each in test_type_comparison.py,
    core/test_expect_table_row_count_to_equal.py and
    core/test_expect_table_row_count_to_be_between.py.
  • pyproject.toml — remove the single 'tests/expectations', entry.
  • scripts/mypy_relaxation_inventory.json — regenerate with the guard's emit mode, never by hand.

How to verify

pip install -r requirements-types.txt -r reqs/requirements-dev-contrib.txt
# remove the 'tests/expectations', line from the exclude list in pyproject.toml, then:
python scripts/mypy_config_guard.py --emit-inventory > scripts/mypy_relaxation_inventory.json
invoke type-check --ci --pretty
pytest tests/expectations/

Before: 134 errors in 16 files; 783 source files checked (at 2c4123fa9).
After: zero errors; 833 checked; pytest tests/expectations/ passing as it does today.

Requirements

  1. When the type-check runs on the merged change, it reports zero errors and the
    'tests/expectations', pattern is gone.
  2. The two truthy-function errors — metrics/test_core.py:457 and :4922, where the first
    conjunct of an assert is a function object and therefore always true — must be corrected so
    every conjunct can actually fail, preserving what each assertion was written to establish.
    Show it by breaking the corrected condition on purpose, watching the test fail, and restoring it.
  3. No test may be weakened: no assertion deleted or made vacuous, no expected value loosened,
    no Any or cast hiding a real mismatch, no new skip or xfail.
  4. pytest tests/expectations/ must pass with the same outcomes as before.
  5. The inventory must be regenerated by python scripts/mypy_config_guard.py --emit-inventory
    in the same change, and the guard must pass.
  6. No file under great_expectations/ may change, and no test function signature may gain
    annotations. If the correct fix looks like it lies in library code, comment here instead.

Out of scope: every exclude pattern outside tests/expectations; the
follow_imports = 'silent' override block; any mypy strictness flag; annotations on test
function signatures.

Notes

  • Second-largest unit in the set, but it is 16 files rather than 50, and the top four hold 91
    of the 134 errors.
  • Start with metrics/conftest.py. Its fixtures feed the rest of the directory, so its three
    errors may be behind more than three of the others.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedIssues we'd love to see community contributions for. Join #contributors-contributing in our Slack!maintenanceready-for-workTriaged, accepted, and specified well enough to start today -- the claiming gate

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions