Skip to content

[ViPPET] Implement API route coverage check for functional tests#2009

Open
ktaube26 wants to merge 4 commits intomainfrom
ktaube/api-route-coverage-check
Open

[ViPPET] Implement API route coverage check for functional tests#2009
ktaube26 wants to merge 4 commits intomainfrom
ktaube/api-route-coverage-check

Conversation

@ktaube26
Copy link
Contributor

@ktaube26 ktaube26 commented Mar 20, 2026

Description

This PR introduces an automated API route coverage check for the VIPPET functional test suite. The check verifies that every HTTP endpoint declared in the live OpenAPI spec (/openapi.json) is exercised at least once during a full test run, preventing newly added or renamed routes from silently falling through without any test coverage.

In addition, functional test coverage is added for API endpoints that were previously missing test cases.

Checklist:

  • I agree to use the APACHE-2.0 license for my code changes.
  • I have not introduced any 3rd party components incompatible with APACHE-2.0.
  • I have not included any company confidential information, trade secret, password or security token.
  • I have performed a self-review of my code.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an automated “API route coverage” safety net to the ViPPET functional test suite by recording all HTTP calls made during the pytest session and verifying they cover every endpoint declared in the live OpenAPI spec. Also expands functional coverage for validation-job endpoints.

Changes:

  • Introduce a new functional test that compares recorded HTTP calls vs /openapi.json routes and fails if any endpoints were never exercised.
  • Update the shared http_client fixture to record every outgoing API request for the session.
  • Add functional coverage for GET /jobs/validation/status and GET /jobs/validation/{job_id} in the pipelines validation test.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tools/visual-pipeline-and-platform-evaluation-tool/vippet/tests/functional/test_z_api_coverage.py New end-of-suite coverage assertion against the live OpenAPI spec, with an exclusion list.
tools/visual-pipeline-and-platform-evaluation-tool/vippet/tests/functional/conftest.py Wraps the session-scoped HTTP client to accumulate (METHOD, URL) pairs for later coverage checking.
tools/visual-pipeline-and-platform-evaluation-tool/vippet/tests/functional/test_pipelines_validate.py Adds calls/assertions for validation job listing and summary endpoints to close coverage gaps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ktaube26 ktaube26 enabled auto-merge (squash) March 20, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants