|
| 1 | +# OpenDataHub-Tests Constitution |
| 2 | + |
| 3 | +This constitution defines the non-negotiable principles and governance rules for the opendatahub-tests repository. It applies to all test development, whether performed by humans or AI assistants. |
| 4 | + |
| 5 | +## Core Principles |
| 6 | + |
| 7 | +### I. Simplicity First |
| 8 | + |
| 9 | +All changes MUST favor the simplest solution that works. Complexity MUST be justified. |
| 10 | + |
| 11 | +- Aim for the simplest solution that works while keeping the code clean |
| 12 | +- Do not prepare code for the future just because it may be useful (YAGNI) |
| 13 | +- Every function, variable, fixture, and test written MUST be used, or else removed |
| 14 | +- Flexible code MUST NOT come at the expense of readability |
| 15 | + |
| 16 | +**Rationale**: The codebase is maintained by multiple teams; simplicity ensures maintainability and reduces cognitive load. |
| 17 | + |
| 18 | +### II. Code Consistency |
| 19 | + |
| 20 | +All changes MUST follow existing code patterns and architecture. |
| 21 | + |
| 22 | +- Follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html) |
| 23 | +- Use pre-commit hooks to enforce style (ruff, mypy, flake8) |
| 24 | +- Use absolute import paths; import specific functions rather than modules |
| 25 | +- Use descriptive names; meaningful names are better than short names |
| 26 | +- Add type annotations to all new code; mypy strict mode is enforced |
| 27 | + |
| 28 | +**Rationale**: Consistent patterns reduce the learning curve and prevent architectural drift. |
| 29 | + |
| 30 | +### III. Test Independence |
| 31 | + |
| 32 | +Each test MUST verify a single aspect of the product and SHOULD be independent of other tests. |
| 33 | + |
| 34 | +- Tests MUST have a clear purpose and be easy to understand |
| 35 | +- Tests MUST be properly documented with docstrings explaining what the test does |
| 36 | +- When test dependencies exist, use pytest-dependency plugin to declare them explicitly |
| 37 | +- Group related tests in classes only when they share fixtures; never group unrelated tests |
| 38 | + |
| 39 | +**Rationale**: Independent tests enable parallel execution, easier debugging, and selective test runs. |
| 40 | + |
| 41 | +### IV. Fixture Discipline |
| 42 | + |
| 43 | +Fixtures MUST do one thing only and follow proper scoping. |
| 44 | + |
| 45 | +- Fixture names MUST be nouns describing what they provide (e.g., `storage_secret` not `create_secret`) |
| 46 | +- Fixtures MUST handle setup and teardown using context managers where appropriate |
| 47 | +- Use the narrowest fixture scope that meets the need (function > class > module > session) |
| 48 | +- Conftest.py files MUST contain fixtures only; no utility functions or constants |
| 49 | +- Use `request.param` with dict structures for parameterized fixtures |
| 50 | + |
| 51 | +**Rationale**: Single-responsibility fixtures are easier to debug, reuse, and compose. |
| 52 | + |
| 53 | +### V. Kubernetes API First |
| 54 | + |
| 55 | +All cluster interactions MUST use openshift-python-wrapper or oc CLI. |
| 56 | + |
| 57 | +- Use [openshift-python-wrapper](https://github.com/RedHatQE/openshift-python-wrapper) for all K8s API calls |
| 58 | +- For missing resources, generate them using class_generator and contribute to wrapper |
| 59 | +- Use oc CLI only when wrapper is not relevant (e.g., must-gather generation) |
| 60 | +- Resource lifecycle MUST be managed via context managers to ensure cleanup |
| 61 | + |
| 62 | +**Rationale**: Consistent API abstraction ensures portability between ODH (upstream) and RHOAI (downstream). |
| 63 | + |
| 64 | +### VI. Locality of Behavior |
| 65 | + |
| 66 | +Keep code close to where it is used. |
| 67 | + |
| 68 | +- Keep functions and fixtures close to where they're used initially |
| 69 | +- Move to shared locations (utilities, common conftest) only when multiple modules need them |
| 70 | +- Avoid creating abstractions prematurely |
| 71 | +- Small, focused changes are preferred unless explicitly asked otherwise |
| 72 | + |
| 73 | +**Rationale**: Locality reduces navigation overhead and makes the impact of changes obvious. |
| 74 | + |
| 75 | +### VII. Security Awareness |
| 76 | + |
| 77 | +All code MUST consider security implications. |
| 78 | + |
| 79 | +- Never filter, log, or expose sensitive information (secrets, tokens, credentials) |
| 80 | +- Avoid running destructive commands without explicit user confirmation |
| 81 | +- Use detect-secrets and gitleaks pre-commit hooks to prevent secret leakage |
| 82 | +- Test code MUST NOT introduce vulnerabilities into the tested systems |
| 83 | + |
| 84 | +**Rationale**: Tests interact with production-like clusters; security lapses can have real consequences. |
| 85 | + |
| 86 | +## Test Development Standards |
| 87 | + |
| 88 | +### Test Documentation |
| 89 | + |
| 90 | +- Every test or test class MUST have a docstring explaining what it tests |
| 91 | +- Docstrings MUST be understandable by engineers from other components, managers, or PMs |
| 92 | +- Use Google-format docstrings |
| 93 | +- Comments are allowed only for complex code blocks (e.g., complex regex) |
| 94 | + |
| 95 | +### Test Markers |
| 96 | + |
| 97 | +- All tests MUST apply relevant markers from pytest.ini |
| 98 | +- Use tier markers (smoke, sanity, tier1, tier2) to indicate test priority |
| 99 | +- Use component markers (model_serving, model_registry, llama_stack) for ownership |
| 100 | +- Use infrastructure markers (gpu, parallel, slow) for execution filtering |
| 101 | + |
| 102 | +### Test Organization |
| 103 | + |
| 104 | +- Tests are organized by component in `tests/<component>/` |
| 105 | +- Each component has its own conftest.py for scoped fixtures |
| 106 | +- Utilities go in `utilities/` with topic-specific modules |
| 107 | +- Runtime manifests go in `utilities/manifests/` |
| 108 | + |
| 109 | +## AI-Assisted Development Guidelines |
| 110 | + |
| 111 | +### Developer Responsibility |
| 112 | + |
| 113 | +Developers are ultimately responsible for all code, regardless of whether AI tools assisted. |
| 114 | + |
| 115 | +- Always assume AI-generated code is unsafe and incorrect until verified |
| 116 | +- Double-check all AI suggestions against project patterns and this constitution |
| 117 | +- AI tools MUST be guided by AGENTS.md (symlink to CLAUDE.md if needed) |
| 118 | + |
| 119 | +### AI Code Generation Rules |
| 120 | + |
| 121 | +- AI MUST follow existing patterns; never introduce new architectural concepts without justification |
| 122 | +- AI MUST NOT add unnecessary complexity or "helpful" abstractions |
| 123 | +- AI-generated tests MUST have proper docstrings and markers |
| 124 | +- AI MUST ask when in doubt about requirements or patterns |
| 125 | + |
| 126 | +### Specification-Driven Development |
| 127 | + |
| 128 | +When adopting AI-driven spec development: |
| 129 | + |
| 130 | +- Specifications MUST be in structured format (YAML/JSON with defined schema) |
| 131 | +- Tests MUST include requirement traceability (Polarion, Jira markers) |
| 132 | +- Docstrings MUST follow Given-When-Then pattern for behavioral clarity |
| 133 | +- Generated tests MUST pass pre-commit checks before review |
| 134 | + |
| 135 | +## Governance |
| 136 | + |
| 137 | +### Constitution Authority |
| 138 | + |
| 139 | +This constitution supersedes all other practices when there is a conflict. All PRs and reviews MUST verify compliance. |
| 140 | + |
| 141 | +### Amendment Process |
| 142 | + |
| 143 | +1. Propose changes via PR to `CONSTITUTION.md` |
| 144 | +2. Changes require review by at least two maintainers |
| 145 | +3. Breaking changes (principle removal/redefinition) require team discussion |
| 146 | +4. All amendments MUST update version and date |
| 147 | + |
| 148 | +### Versioning Policy |
| 149 | + |
| 150 | +- MAJOR: Backward-incompatible governance/principle removals or redefinitions |
| 151 | +- MINOR: New principle/section added or materially expanded guidance |
| 152 | +- PATCH: Clarifications, wording, typo fixes, non-semantic refinements |
| 153 | + |
| 154 | +### Compliance Review |
| 155 | + |
| 156 | +- All PRs MUST be verified against constitution principles |
| 157 | +- Pre-commit hooks enforce code quality standards |
| 158 | +- CI (tox) validates test structure and typing |
| 159 | +- Two reviewers required; verified label required before merge |
| 160 | + |
| 161 | +### Guidance Reference |
| 162 | + |
| 163 | +For development runtime guidance, consult: |
| 164 | +- [AGENTS.md](./AGENTS.md) for AI assistant instructions |
| 165 | +- [DEVELOPER_GUIDE.md](./docs/DEVELOPER_GUIDE.md) for contribution details |
| 166 | +- [STYLE_GUIDE.md](./docs/STYLE_GUIDE.md) for code style |
| 167 | + |
| 168 | +**Version**: 1.0.0 | **Ratified**: 2026-01-08 | **Last Amended**: 2026-01-08 |
0 commit comments