Skip to content

Add snap packages to metadata collected in JSON report (new)#2396

Merged
Hook25 merged 12 commits into
mainfrom
2191-more-metadata-in-json-report
Mar 19, 2026
Merged

Add snap packages to metadata collected in JSON report (new)#2396
Hook25 merged 12 commits into
mainfrom
2191-more-metadata-in-json-report

Conversation

@pieqq

@pieqq pieqq commented Mar 17, 2026

Copy link
Copy Markdown
Collaborator

Description

Add snap packages to metadata collected by system_information.

This required to move the snap_utils support modules from checkbox_support to checkbox_ng.support, with the usual dance:

  • leaving "empty" modules in checkbox_support.snap_utils that redirect to the ones in checkbox_ng.support.snap_utils
  • upgrading the MANIFEST.in files so that test files can be found

The device_info script is updated with a new snaps subcommand.

system_information Collector added to make use of the new device_info subcommand. The Collector version was bumped to 4.

Note: the gathered metadata is more complete, since every information about each snap (as returned by the Snapd internals) is kept. In contrast, the previous data only included a subset of the data.

Resolved issues

https://warthogs.atlassian.net/browse/CHECKBOX-2191

Documentation

Tests

  • Unit tests have been run locally

Virtual env testing

A sample Checkbox run was done. The associated submission.json includes a new snaps subsection in the system_information section that looks like:

`system_information` section from the `submission.json` report
    "snaps": {
        "tool_version": "7.0.1.dev25+g9a3965953.d20260311",
        "success": true,
        "outputs": {
                ...
                {
                    "apps": [
                        {
                            "activators": [
                                {
                                    "Active": false,
                                    "Enabled": true,
                                    "Name": "firmware-notifier",
                                    "Type": "timer",
                                    "active": false,
                                    "enabled": true,
                                    "name": "firmware-notifier",
                                    "type": "timer"
                                }
                            ],
                            "daemon": "simple",
                            "daemon-scope": "user",
                            "enabled": true,
                            "name": "firmware-notifier",
                            "snap": "firmware-updater"
                        },
                        {
                            "desktop-file": "/var/lib/snapd/desktop/applications/firmware-updater_firmware-updater.desktop",
                            "name": "firmware-updater",
                            "snap": "firmware-updater"
                        },
                        {
                            "activators": [
                                {
                                    "Active": true,
                                    "Enabled": true,
                                    "Name": "com.canonical.firmware_updater",
                                    "Type": "dbus",
                                    "active": true,
                                    "enabled": true,
                                    "name": "com.canonical.firmware_updater",
                                    "type": "dbus"
                                }
                            ],
                            "daemon": "dbus",
                            "daemon-scope": "user",
                            "desktop-file": "/var/lib/snapd/desktop/applications/firmware-updater_firmware-updater-app.desktop",
                            "enabled": true,
                            "name": "firmware-updater-app",
                            "snap": "firmware-updater"
                        }
                    ],
                    "base": "core22",
                    "channel": "1/stable",
                    "confinement": "strict",
                    "contact": "https://github.com/canonical/firmware-updater/",
                    "description": "Update Firmware",
                    "developer": "canonical",
                    "devmode": false,
                    "grade": "stable",
                    "icon": "/v2/icons/firmware-updater/icon",
                    "id": "EI0D1KHjP8XiwMZKqSjuh6W8zvcowUVP",
                    "ignore-validation": false,
                    "install-date": "2026-03-03T15:25:48.539611864+01:00",
                    "installed-size": 17244160,
                    "jailmode": false,
                    "license": "GPL-3.0+",
                    "links": {
                        "contact": [
                            "https://github.com/canonical/firmware-updater/"
                        ],
                        "website": [
                            "https://github.com/canonical/firmware-updater/"
                        ]
                    },
                    "media": [
                        {
                            "height": 256,
                            "type": "icon",
                            "url": "https://dashboard.snapcraft.io/site_media/appmedia/2022/01/system-component-firmware.png",
                            "width": 256
                        },
                        {
                            "height": 1275,
                            "type": "screenshot",
                            "url": "https://dashboard.snapcraft.io/site_media/appmedia/2022/02/firmware-updater.png",
                            "width": 752
                        }
                    ],
                    "mounted-from": "/var/lib/snapd/snaps/firmware-updater_216.snap",
                    "name": "firmware-updater",
                    "private": false,
                    "publisher": {
                        "display-name": "Canonical",
                        "id": "canonical",
                        "username": "canonical",
                        "validation": "verified"
                    },
                    "revision": "216",
                    "status": "active",
                    "summary": "Firmware Updater",
                    "title": "firmware-updater",
                    "tracking-channel": "1/stable/ubuntu-24.04",
                    "type": "app",
                    "version": "0+git.e608b29",
                    "website": "https://github.com/canonical/firmware-updater/"
                }
            ],
            "stderr": ""
        }
    }

Debian package testing

I built the Debian packages in a LXC container, then installed them and ran checkbox-cli with the usb-automated test plan.

The LXC container has the following snaps installed:

Name    Version   Rev    Tracking       Publisher     Notes
core24  20260211  1499   latest/stable  canonical✓    base
helix   25.07.1   120    latest/stable  lauren-brock  classic
snapd   2.73      25935  latest/stable  canonical✓    snapd

It generated the following submission.json report:

submission_debian_packages.zip

Snap testing

Following the latest commits (9fb88b2 and e939572), I built a checkbox24 snap locally and tested it using @Hook25 spread suite on a UC24 VM.

It generated the following submission.json:

submission.json

As you can see, the new snaps section is included and working.

@pieqq pieqq marked this pull request as draft March 17, 2026 15:34
@codecov

codecov Bot commented Mar 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.10290% with 155 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.10%. Comparing base (3f533aa) to head (e939572).
⚠️ Report is 121 commits behind head on main.

Files with missing lines Patch % Lines
...heckbox-ng/checkbox_ng/support/snap_utils/snapd.py 54.28% 79 Missing and 1 partial ⚠️
...eckbox-ng/checkbox_ng/support/snap_utils/system.py 0.00% 58 Missing and 2 partials ⚠️
...eckbox-ng/checkbox_ng/support/snap_utils/config.py 86.15% 8 Missing and 1 partial ⚠️
checkbox-ng/checkbox_ng/support/device_info.py 75.00% 1 Missing ⚠️
...ckbox-ng/checkbox_ng/support/snap_utils/asserts.py 97.43% 0 Missing and 1 partial ⚠️
...box-ng/plainbox/impl/session/system_information.py 80.00% 1 Missing ⚠️
...box-support/checkbox_support/snap_utils/asserts.py 0.00% 1 Missing ⚠️
...kbox-support/checkbox_support/snap_utils/config.py 0.00% 1 Missing ⚠️
...ckbox-support/checkbox_support/snap_utils/snapd.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2396      +/-   ##
==========================================
+ Coverage   57.76%   58.10%   +0.33%     
==========================================
  Files         467      474       +7     
  Lines       47299    47739     +440     
  Branches     8419     8486      +67     
==========================================
+ Hits        27321    27737     +416     
- Misses      19092    19116      +24     
  Partials      886      886              
Flag Coverage Δ
checkbox-ng 75.84% <59.20%> (+0.16%) ⬆️
checkbox-support 67.30% <25.00%> (+0.45%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

pieqq added 12 commits March 18, 2026 20:50
This is required so that scripts that make use of
`checkbox_support.snap_utils` can still call it, and are served the
modules they expect.
checkbox22 has "python3-requests-unixsocket" as a stage-package in
checkbox-support. checkbox24 removed "python3-requests-unixsocket" from
checkbox-support stage-packages. When checkbox-ng is built, it needs
"requests" library (specified in pyproject.toml). pip tries to
upgrade/install requests which depends on urllib3, but urllib3 2.0.7
comes from the Debian package system (not from pip). Without the
requests-unixsocket package in stage, pip cannot find the necessary
dependencies and tries to uninstall the system-provided urllib3 2.0.7.

Adding PIP_IGNORE_INSTALLED should prevent pip from tripping over an
already installed system package.
The python3-requests-unixsocket requirement has moved from
checkbox-support to checkbox-ng
In Snapcraft documentation about the python plugin[1], it is told that
"If the source contains a setup.py or pyproject.toml file, those files
are used to install the dependencies specified by the package itself."

Because of this, the request_unixsocket2 package is being pulled, and it
requires urllib3 2.4 or highed. However, the Debian stage package
python3-requests-oauthlib also pulls urllib3, but it's the version from
the Debian archives, which is 2.0.7. Because of this, pip tries to
uninstall the old version and fails (because it's a Debian install).

Replacing the python3-requests-oauthlib stage package with
requests-oauthlib python package fixes the build issue.

[1]:
https://documentation.ubuntu.com/snapcraft/latest/reference/plugins/python_plugin/#how-it-works
@pieqq pieqq force-pushed the 2191-more-metadata-in-json-report branch from 71d51cd to e939572 Compare March 18, 2026 19:56
@pieqq pieqq marked this pull request as ready for review March 19, 2026 07:29
@Hook25 Hook25 merged commit d317418 into main Mar 19, 2026
181 of 186 checks passed
@Hook25 Hook25 deleted the 2191-more-metadata-in-json-report branch March 19, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants