Skip to content

Change certification level status (BugFix) #2424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
clairlin53 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Change certification level status (BugFix) #2424

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0799979
update_by_overrides
clairlin53 Apr 16, 2026
730fe3e
Remove redundant overrides from top-level 26.04 plan
clairlin53 Apr 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions providers/base/units/audio/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,13 @@ nested_part:
audio-cert-manual
after-suspend-audio-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::audio/channels
apply blocker to com.canonical.certification::audio/external-linein
apply blocker to com.canonical.certification::audio/external-lineout
apply blocker to com.canonical.certification::after-suspend-audio/channels
apply blocker to com.canonical.certification::after-suspend-audio/external-linein
apply blocker to com.canonical.certification::after-suspend-audio/external-lineout

id: audio-base-26-04-automated
unit: test plan
Expand All @@ -201,3 +208,10 @@ include:
nested_part:
audio-cert-automated
after-suspend-audio-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::audio/detect_sinks
apply blocker to com.canonical.certification::audio/detect_sources
apply blocker to com.canonical.certification::audio/alsa_record_playback_automated
apply blocker to com.canonical.certification::after-suspend-audio/detect_sinks
apply blocker to com.canonical.certification::after-suspend-audio/detect_sources
apply blocker to com.canonical.certification::after-suspend-audio/alsa_record_playback_automated
3 changes: 3 additions & 0 deletions providers/base/units/bluetooth/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,3 +211,6 @@ include:
nested_part:
bluetooth-cert-automated
after-suspend-bluetooth-cert-automated
certification_status_overrides:
apply non-blocker to com.canonical.certification::bluetooth4/beacon_eddystone_url_.*
apply non-blocker to com.canonical.certification::after-suspend-bluetooth4/beacon_eddystone_url_.*
13 changes: 13 additions & 0 deletions providers/base/units/fingerprint/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,19 @@ nested_part:
fingerprint-cert-manual
after-suspend-fingerprint-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::fingerprint/detect
apply blocker to com.canonical.certification::fingerprint/enroll
apply blocker to com.canonical.certification::fingerprint/verify-no-match
apply blocker to com.canonical.certification::fingerprint/verify-match
apply blocker to com.canonical.certification::fingerprint/unlock
apply blocker to com.canonical.certification::fingerprint/delete
apply blocker to com.canonical.certification::after-suspend-fingerprint/detect
apply blocker to com.canonical.certification::after-suspend-fingerprint/enroll
apply blocker to com.canonical.certification::after-suspend-fingerprint/verify-no-match
apply blocker to com.canonical.certification::after-suspend-fingerprint/verify-match
apply blocker to com.canonical.certification::after-suspend-fingerprint/unlock
apply blocker to com.canonical.certification::after-suspend-fingerprint/delete

id: fingerprint-base-26-04-automated
unit: test plan
Expand Down
1 change: 1 addition & 0 deletions providers/base/units/graphics/jobs.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,7 @@ template-resource: graphics_card
plugin: shell
category_id: com.canonical.plainbox::graphics
id: graphics/{index}_vkcube_{product_slug}
template-id: graphics/index_vkcube_product_slug
requires:
executable.name == 'vkcube'
dmi.sane_product in ['portable', 'not-portable']
Expand Down
9 changes: 9 additions & 0 deletions providers/base/units/graphics/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,11 @@ nested_part:
after-suspend-graphics-gpu-cert-manual
after-suspend-monitor-gpu-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::graphics/1_cycle_resolution_.*
apply blocker to com.canonical.certification::graphics/2_cycle_resolution_.*
apply blocker to com.canonical.certification::after-suspend-graphics/1_cycle_resolution_.*
apply blocker to com.canonical.certification::after-suspend-graphics/2_cycle_resolution_.*

id: graphic-base-26-04-automated
unit: test plan
Expand All @@ -369,6 +374,10 @@ include:
nested_part:
graphics-gpu-cert-automated
after-suspend-graphics-gpu-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::graphics/1_vkcube_.*
apply blocker to com.canonical.certification::graphics/2_vkcube_.*
apply blocker to com.canonical.certification::graphics/vulkaninfo.*

id: opencl-core-test-plan
unit: test plan
Expand Down
5 changes: 5 additions & 0 deletions providers/base/units/keys/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,11 @@ nested_part:
keys-cert-manual
after-suspend-keys-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::keys/fn-lock
apply blocker to com.canonical.certification::keys/power-button-suspend
apply blocker to com.canonical.certification::keys/sleep
apply blocker to com.canonical.certification::after-suspend-keys/fn-lock

id: keys-base-26-04-automated
unit: test plan
Expand Down
3 changes: 3 additions & 0 deletions providers/base/units/led/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,9 @@ nested_part:
led-cert-manual
after-suspend-led-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::led/fn
apply blocker to com.canonical.certification::after-suspend-led/fn

id: led-base-26-04-automated
unit: test plan
Expand Down
13 changes: 13 additions & 0 deletions providers/base/units/monitor/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,19 @@ nested_part:
monitor-gpu-cert-manual
after-suspend-monitor-gpu-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::power-management/light_sensor
apply blocker to com.canonical.certification::monitor/1_thunderbolt3_.*
apply blocker to com.canonical.certification::audio/1_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::monitor/2_thunderbolt3_.*
apply blocker to com.canonical.certification::audio/2_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::thunderbolt3/daisy-chain
apply blocker to com.canonical.certification::after-suspend-power-management/light_sensor
apply blocker to com.canonical.certification::after-suspend-monitor/1_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-audio/1_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-monitor/2_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-audio/2_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-thunderbolt3/daisy-chain

id: monitor-base-26-04-automated
unit: test plan
Expand Down
4 changes: 4 additions & 0 deletions providers/base/units/stress/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,3 +337,7 @@ nested_part:
suspend-cycles-stress-test
stress-warmboot-coldboot-automated
stress-pm-graph
certification_status_overrides:
apply blocker to com.canonical.certification::warm-boot-loop-.*
apply blocker to com.canonical.certification::cold-boot-loop-.*
apply blocker to com.canonical.certification::ethernet/iperf3_.*
2 changes: 2 additions & 0 deletions providers/base/units/suspend/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,3 +141,5 @@ include:
nested_part:
before-suspend-reference-cert-automated
after-suspend-reference-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::suspend/validate_suspend_status
3 changes: 3 additions & 0 deletions providers/base/units/touchpad/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ nested_part:
touchpad-cert-manual
after-suspend-touchpad-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::touchpad/palm-rejection
apply blocker to com.canonical.certification::after-suspend-touchpad/palm-rejection

id: touchpad-base-26-04-automated
unit: test plan
Expand Down
3 changes: 3 additions & 0 deletions providers/base/units/touchscreen/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,9 @@ nested_part:
touchscreen-cert-manual
after-suspend-touchscreen-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::touchscreen/multitouch-rotate
apply blocker to com.canonical.certification::after-suspend-touchscreen/multitouch-rotate

id: touchscreen-base-26-04-automated
unit: test plan
Expand Down
8 changes: 8 additions & 0 deletions providers/base/units/usb/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,11 @@ nested_part:
usb3-cert-automated
after-suspend-usb-cert-automated
after-suspend-usb3-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::usb/storage-detect
apply blocker to com.canonical.certification::usb/storage-preinserted-.*
apply blocker to com.canonical.certification::after-suspend-usb/storage-detect
apply blocker to com.canonical.certification::after-suspend-usb/storage-preinserted-.*

id: usb-c-base-26-04-manual
unit: test plan
Expand All @@ -420,6 +425,9 @@ nested_part:
usb-c-cert-manual
after-suspend-usb-c-cert-manual
info-attachment-cert-full
certification_status_overrides:
apply blocker to com.canonical.certification::usb-c/c-to-ethernet-adapter-insert
apply blocker to com.canonical.certification::after-suspend-usb-c/c-to-ethernet-adapter-insert

id: usb-c-base-26-04-automated
unit: test plan
Expand Down
2 changes: 2 additions & 0 deletions providers/base/units/wireless/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -478,6 +478,8 @@ include:
nested_part:
wireless-cert-automated
after-suspend-wireless-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::wireless/detect

id: wireless-netplan-base-26-04-automated
unit: test plan
Expand Down
7 changes: 7 additions & 0 deletions providers/base/units/wwan/test-plan.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,10 @@ include:
nested_part:
wwan-cert-automated
after-suspend-wwan-cert-automated
certification_status_overrides:
apply blocker to com.canonical.certification::wwan/detect
apply blocker to com.canonical.certification::wwan/check-sim-present-manufacturer-model-hw_id-auto
apply blocker to com.canonical.certification::wwan/gsm-reconnection-.*-auto
apply blocker to com.canonical.certification::after-suspend-wwan/detect
apply blocker to com.canonical.certification::after-suspend-wwan/check-sim-present-manufacturer-model-hw_id-auto
apply blocker to com.canonical.certification::after-suspend-wwan/gsm-reconnection-.*-auto
75 changes: 75 additions & 0 deletions providers/certification-client/units/client-cert-desktop-26-04.pxu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,53 @@ nested_part:
optical-base-26-04-manual
power-management-base-26-04-manual
info-attachment-cert-manual
certification_status_overrides:
apply blocker to com.canonical.certification::audio/channels
apply blocker to com.canonical.certification::audio/external-linein
apply blocker to com.canonical.certification::audio/external-lineout
apply blocker to com.canonical.certification::after-suspend-audio/channels
apply blocker to com.canonical.certification::after-suspend-audio/external-linein
apply blocker to com.canonical.certification::after-suspend-audio/external-lineout
apply blocker to com.canonical.certification::fingerprint/detect
apply blocker to com.canonical.certification::fingerprint/enroll
apply blocker to com.canonical.certification::fingerprint/verify-no-match
apply blocker to com.canonical.certification::fingerprint/verify-match
apply blocker to com.canonical.certification::fingerprint/unlock
apply blocker to com.canonical.certification::fingerprint/delete
apply blocker to com.canonical.certification::after-suspend-fingerprint/detect
apply blocker to com.canonical.certification::after-suspend-fingerprint/enroll
apply blocker to com.canonical.certification::after-suspend-fingerprint/verify-no-match
apply blocker to com.canonical.certification::after-suspend-fingerprint/verify-match
apply blocker to com.canonical.certification::after-suspend-fingerprint/unlock
apply blocker to com.canonical.certification::after-suspend-fingerprint/delete
apply blocker to com.canonical.certification::graphics/1_cycle_resolution_.*
apply blocker to com.canonical.certification::graphics/2_cycle_resolution_.*
apply blocker to com.canonical.certification::after-suspend-graphics/1_cycle_resolution_.*
apply blocker to com.canonical.certification::after-suspend-graphics/2_cycle_resolution_.*
apply blocker to com.canonical.certification::keys/fn-lock
apply blocker to com.canonical.certification::keys/power-button-suspend
apply blocker to com.canonical.certification::keys/sleep
apply blocker to com.canonical.certification::after-suspend-keys/fn-lock
apply blocker to com.canonical.certification::led/fn
apply blocker to com.canonical.certification::after-suspend-led/fn
apply blocker to com.canonical.certification::power-management/light_sensor
apply blocker to com.canonical.certification::monitor/1_thunderbolt3_.*
apply blocker to com.canonical.certification::audio/1_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::monitor/2_thunderbolt3_.*
apply blocker to com.canonical.certification::audio/2_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::thunderbolt3/daisy-chain
apply blocker to com.canonical.certification::after-suspend-power-management/light_sensor
apply blocker to com.canonical.certification::after-suspend-monitor/1_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-audio/1_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-monitor/2_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-audio/2_playback_thunderbolt3_.*
apply blocker to com.canonical.certification::after-suspend-thunderbolt3/daisy-chain
apply blocker to com.canonical.certification::touchpad/palm-rejection
apply blocker to com.canonical.certification::after-suspend-touchpad/palm-rejection
apply blocker to com.canonical.certification::touchscreen/multitouch-rotate
apply blocker to com.canonical.certification::after-suspend-touchscreen/multitouch-rotate
apply blocker to com.canonical.certification::usb-c/c-to-ethernet-adapter-insert
apply blocker to com.canonical.certification::after-suspend-usb-c/c-to-ethernet-adapter-insert
Copy link
Copy Markdown
Collaborator

@pieqq pieqq Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment below, but I don't think this is needed, unless you can provide a way to reproduce in #2479.

Copy link
Copy Markdown
Contributor Author

@clairlin53 clairlin53 Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment below, but I don't think this is needed, unless you can provide a way to reproduce in #2479.

"Hi @pieqq,

I've updated the PR to remove the overrides from the top-level 26.04.pxu.

Interestingly, I encountered a blocking behavior in my previous test where the feature test plan overrides didn't seem to be applied by the top plan.
I re-verified the test in a clean environment that the overrides in the feature test plan are working as expected without adding override in the top level test plan.

I've removed the redundant code. Please help to review again. Thanks!



id: client-cert-desktop-26-04-automated
Expand Down Expand Up @@ -95,6 +142,30 @@ nested_part:
bootstrap_include:
device
graphics_card
certification_status_overrides:
apply blocker to com.canonical.certification::audio/detect_sinks
apply blocker to com.canonical.certification::audio/detect_sources
apply blocker to com.canonical.certification::audio/alsa_record_playback_automated
apply blocker to com.canonical.certification::after-suspend-audio/detect_sinks
apply blocker to com.canonical.certification::after-suspend-audio/detect_sources
apply blocker to com.canonical.certification::after-suspend-audio/alsa_record_playback_automated
apply non-blocker to com.canonical.certification::bluetooth4/beacon_eddystone_url_.*
apply non-blocker to com.canonical.certification::after-suspend-bluetooth4/beacon_eddystone_url_.*
apply blocker to com.canonical.certification::graphics/1_vkcube_.*
apply blocker to com.canonical.certification::graphics/2_vkcube_.*
apply blocker to com.canonical.certification::graphics/vulkaninfo.*
apply blocker to com.canonical.certification::suspend/validate_suspend_status
apply blocker to com.canonical.certification::usb/storage-detect
apply blocker to com.canonical.certification::usb/storage-preinserted-.*
apply blocker to com.canonical.certification::after-suspend-usb/storage-detect
apply blocker to com.canonical.certification::after-suspend-usb/storage-preinserted-.*
apply blocker to com.canonical.certification::wireless/detect
apply blocker to com.canonical.certification::wwan/detect
apply blocker to com.canonical.certification::wwan/check-sim-present-manufacturer-model-hw_id-auto
apply blocker to com.canonical.certification::wwan/gsm-reconnection-.*-auto
apply blocker to com.canonical.certification::after-suspend-wwan/detect
apply blocker to com.canonical.certification::after-suspend-wwan/check-sim-present-manufacturer-model-hw_id-auto
apply blocker to com.canonical.certification::after-suspend-wwan/gsm-reconnection-.*-auto
Copy link
Copy Markdown
Collaborator

@pieqq pieqq Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I've explained in #2479 (comment), I've ran checkbox-cli expand --format json on this test plan with and without this section, and I get the same result.

When using list-bootstrapped, it's a bit more complicated to parse, but I basically get the same thing (at least, the few jobs I checked all had blocker for the certification status).

So I don't think this is needed.



id: client-cert-desktop-26-04-stress
Expand All @@ -106,3 +177,7 @@ include:
nested_part:
submission-cert-automated
stress-base-26-04-automated
certification_status_overrides:
apply blocker to com.canonical.certification::warm-boot-loop-.*
apply blocker to com.canonical.certification::cold-boot-loop-.*
apply blocker to com.canonical.certification::ethernet/iperf3_.*
Copy link
Copy Markdown
Collaborator

@pieqq pieqq Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above comment. Probably not needed, unless you can provide some clear steps to reproduce in #2479

Loading