Skip to content

Add Hypothesis property-based tests for substitution logic#1424

Open
adamtheturtle wants to merge 3 commits intomainfrom
adamtheturtle/add-hypothesis-tests
Open

Add Hypothesis property-based tests for substitution logic#1424
adamtheturtle wants to merge 3 commits intomainfrom
adamtheturtle/add-hypothesis-tests

Conversation

@adamtheturtle
Copy link
Owner

@adamtheturtle adamtheturtle commented Mar 13, 2026

Summary

  • Add hypothesis dependency for property-based testing
  • Add 8 property-based tests covering _apply_substitutions and flag behavior:
    • Empty substitutions is identity
    • All placeholder occurrences are replaced
    • Text without placeholders is unchanged
    • No leftover placeholders after substitution
    • MyST {{ }} delimiters work correctly
    • Multiple delimiter pairs work in one call
    • :nosubstitutions: prevents replacement regardless of config
    • :substitutions: forces replacement regardless of config

Test plan

  • All 47 tests pass (8 new + 39 existing)
  • 100% code coverage maintained
  • All pre-commit hooks pass (mypy, pyright, ruff, interrogate, etc.)

Closes #1423

🤖 Generated with Claude Code


Note

Low Risk
Low risk: only adds a dev-only testing dependency and new property-based tests; no runtime code or substitution logic is modified.

Overview
Adds hypothesis to the dev dependency set and introduces a new tests/test_hypothesis.py suite of property-based tests.

The new tests stress _apply_substitutions across randomized inputs (identity cases, full replacement, MyST {{ }} support, and mixed delimiter pairs) and verify :nosubstitutions:/:substitutions: flags override substitutions_default_enabled when building a minimal Sphinx project.

Written by Cursor Bugbot for commit 5821c2c. This will update automatically on new commits. Configure here.

adamtheturtle and others added 3 commits March 13, 2026 15:49
Closes #1423

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.


@settings(
suppress_health_check=[HealthCheck.function_scoped_fixture],
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing deadline=None causes flaky Sphinx build tests

Medium Severity

The @settings for both Sphinx build tests suppresses HealthCheck.function_scoped_fixture but doesn't set deadline=None. Hypothesis defaults to a 200ms deadline per example. Each example creates a temp directory, writes files, initializes a SphinxTestApp, runs a full build, and reads HTML output — very likely exceeding 200ms, especially in resource-constrained CI environments. This can cause intermittent DeadlineExceeded failures.

Additional Locations (1)
Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider adding Hypothesis for property-based testing of substitution logic

1 participant