Skip to content

Add 'disabled' field to IntegrationTestScenario spec#1502

Open
jangel97 wants to merge 1 commit into
konflux-ci:mainfrom
jangel97:add-disabled-field-to-its
Open

Add 'disabled' field to IntegrationTestScenario spec#1502
jangel97 wants to merge 1 commit into
konflux-ci:mainfrom
jangel97:add-disabled-field-to-its

Conversation

@jangel97

@jangel97 jangel97 commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

There is currently no first-class way to disable an IntegrationTestScenario without deleting the resource. Users who want to temporarily disable a test must either remove the ITS entirely or rely on workarounds such as setting a non-matching context name (e.g. disabled), which depends on the fallthrough return false in IsContextValidForSnapshot:

https://github.com/konflux-ci/integration-service/blob/main/gitops/snapshot.go#L1233

While this works, it is not explicit and relies on implementation details.

Adding a field like spec.disabled would make the intent clear and improve operability. This pattern is common in Kubernetes APIs (e.g. CronJob.spec.suspend) to temporarily stop executions without deleting or modifying the resource configuration. It would also be discoverable via kubectl explain, making the behavior clearer to users.

Example

An IntegrationTestScenario could be temporarily disabled by setting spec.disabled: true:

apiVersion: appstudio.redhat.com/v1beta2
kind: IntegrationTestScenario
metadata:
  name: example-test
spec:
  disabled: true
  application: my-app
  contexts:
    - name: component_my-component

When spec.disabled is set to true, the Integration Service ignores the scenario during snapshot evaluation. Setting it back to false (or omitting the field) re-enables it without deleting or modifying contexts.

Summary

  • Adds a disabled boolean field to IntegrationTestScenarioSpec (default: false)
  • When disabled: true, the scenario is skipped in IsScenarioApplicableToSnapshotsContext before context matching
  • Provides a first-class mechanism to temporarily disable integration tests without deleting the resource
  • Includes CRD regeneration and test coverage

Motivation

Currently there is no official way to disable an ITS without deleting it. Users resort to workarounds such as using unrecognized context names (e.g. disabled), which rely on the fallthrough return false in IsContextValidForSnapshot.

A dedicated disabled field makes this behavior explicit and easier to operate.

Test plan

  • Added test: disabled scenario with no contexts is filtered out
  • Added test: disabled scenario with explicit contexts is still filtered out
  • Existing context filtering tests updated to include disabled scenario in the list (counts unchanged)
  • All 110 gitops tests pass

Closes #1501

@snyk-io

snyk-io Bot commented Mar 16, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@codecov-commenter

codecov-commenter commented Mar 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.22%. Comparing base (9b742d0) to head (305475f).

Files with missing lines Patch % Lines
gitops/snapshot.go 0.00% 1 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (9b742d0) and HEAD (305475f). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (9b742d0) HEAD (305475f)
unit-tests 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1502       +/-   ##
===========================================
- Coverage   65.12%   42.22%   -22.91%     
===========================================
  Files          56       53        -3     
  Lines        7722     7335      -387     
===========================================
- Hits         5029     3097     -1932     
- Misses       2085     3738     +1653     
+ Partials      608      500      -108     
Flag Coverage Δ
e2e-tests 42.22% <0.00%> (?)
unit-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jangel97 jangel97 force-pushed the add-disabled-field-to-its branch from 416c539 to 1799378 Compare March 17, 2026 08:49
@hongweiliu17

Copy link
Copy Markdown
Contributor

/ok-to-test

@hongweiliu17

Copy link
Copy Markdown
Contributor

The go linter issue will be fixed by #1511

@hongweiliu17

Copy link
Copy Markdown
Contributor

The code looks good to me. And need a rebase after merging #1511. Thanks!

@kasemAlem

Copy link
Copy Markdown
Contributor

/retest

@jangel97 jangel97 force-pushed the add-disabled-field-to-its branch from 1799378 to 305475f Compare March 24, 2026 11:04
@jangel97

Copy link
Copy Markdown
Contributor Author

@hongweiliu17 @kasemAlem I will wait for #1511 to be merged and then rebase this one. Thank you for looking into it!

@kasemAlem

Copy link
Copy Markdown
Contributor

/retest

Add a boolean 'disabled' field to the
IntegrationTestScenarioSpec that allows users to skip
a scenario without deleting it. When set to true, the
scenario is filtered out before any context matching
occurs.

This provides a first-class mechanism to temporarily
disable integration tests, replacing the workaround
of using unrecognized context names.

Closes konflux-ci#1501

Signed-off-by: Jose Angel Morena <jmorenas@redhat.com>
@jangel97 jangel97 force-pushed the add-disabled-field-to-its branch from 305475f to ead23ae Compare March 26, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add 'disabled' field to IntegrationTestScenario spec

4 participants