Add webgl conformance test (New) #2115

hanhsuan · 2025-09-17T12:05:59Z

Description

In order to close test coverage gap for Ubuntu GA Hardware Feature Matrix, the new WebGL conformance test is added in this PR.

WEBGL_README.md: The detail of WebGL conformance test in checkbox
file_watcher.py: A simple file watcher implementation that utilize the inotify
Environment WEBGL_CONFORMANCE_TEST_URL: for pointing to the self-hosted WebGL conformance test server
Manifest entry: has_webgl_support:
browser_resource: Simply echo firefox, chromium and google-chrome
test plan: webgl-conformance-test
test cases: graphics/webgl-conformance-test-.* and after-suspend-graphics/webgl-conformance-test-.*
Each browser will take more than one hour to finish the WebGL conformance test on Normal PCs.

Resolved issues

https://warthogs.atlassian.net/browse/OEMQA-6284

Documentation

ref: WEBGL_README.md

Tests

https://certification.canonical.com/hardware/202507-36996/submission/450107/

codecov · 2025-09-17T12:06:14Z

Codecov Report

❌ Patch coverage is 92.08333% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.85%. Comparing base (a65231d) to head (bca26d4).

Files with missing lines	Patch %	Lines
providers/base/bin/webgl_conformance_test.py	84.84%	13 Missing and 2 partials ⚠️
...x-support/checkbox_support/helpers/file_watcher.py	92.30%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2115      +/-   ##
==========================================
+ Coverage   53.63%   53.85%   +0.21%     
==========================================
  Files         401      404       +3     
  Lines       43116    43356     +240     
  Branches     7990     8032      +42     
==========================================
+ Hits        23127    23348     +221     
- Misses      19174    19192      +18     
- Partials      815      816       +1

Flag	Coverage Δ
checkbox-support	`66.01% <97.16%> (+0.72%)`	⬆️
provider-base	`31.22% <83.83%> (+0.31%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tomli380576

lgtm, I was able to setup a local server and run the tests

providers/base/bin/webgl_conformance_test.py

Copilot

Pull Request Overview

This PR adds WebGL conformance testing capabilities to Checkbox to close a test coverage gap for Ubuntu GA Hardware Feature Matrix. The implementation includes a custom file watcher utility using inotify, test job definitions for multiple browsers (Firefox, Chromium, Google Chrome), and comprehensive test coverage.

Key Changes:

New file watcher implementation using inotify for monitoring test result files
WebGL conformance test jobs for Firefox, Chromium, and Google Chrome with browser-specific launch options
Test plans for normal and after-suspend WebGL conformance testing

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
providers/base/units/graphics/test-plan.pxu	Adds two test plans for WebGL conformance testing (normal and after-suspend)
providers/base/units/graphics/resource.pxu	Defines browser_resource job that lists browsers to test
providers/base/units/graphics/manifest.pxu	Adds has_webgl_support manifest entry
providers/base/units/graphics/jobs.pxu	Defines template job for WebGL conformance tests across browsers
providers/base/units/graphics/WEBGL_README.md	Documentation for WebGL conformance testing setup and requirements
providers/base/tests/test_webgl_conformance_test.py	Unit tests for WebGL conformance test script
checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py	Unit tests for FileWatcher utility
checkbox-support/checkbox_support/helpers/file_watcher.py	FileWatcher implementation using inotify system calls

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-21T08:06:34Z

checkbox-support/checkbox_support/helpers/file_watcher.py

+import os
+
+
+class InotfiyEvent:


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Suggested change

class InotfiyEvent:

class InotifyEvent:

Copilot · 2025-11-21T08:06:35Z

checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py

@@ -0,0 +1,149 @@
+from checkbox_support.helpers.file_watcher import FileWatcher, InotfiyEvent


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Copilot · 2025-11-21T08:06:35Z

checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py

+from checkbox_support.helpers.file_watcher import FileWatcher, InotfiyEvent
+from unittest.mock import MagicMock, patch
+import unittest
+
+
+class TestInotifyEvent(unittest.TestCase):
+    """
+    Test the InotfiyEvent data class.
+    """
+
+    def test_inotify_event_creation(self):
+        event = InotfiyEvent(


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Suggested change

from checkbox_support.helpers.file_watcher import FileWatcher, InotfiyEvent

from unittest.mock import MagicMock, patch

import unittest

class TestInotifyEvent(unittest.TestCase):

"""

Test the InotfiyEvent data class.

"""

def test_inotify_event_creation(self):

event = InotfiyEvent(

from checkbox_support.helpers.file_watcher import FileWatcher, InotifyEvent

from unittest.mock import MagicMock, patch

import unittest

class TestInotifyEvent(unittest.TestCase):

"""

Test the InotifyEvent data class.

"""

def test_inotify_event_creation(self):

event = InotifyEvent(

Copilot · 2025-11-21T08:06:35Z

checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py

+    """
+
+    def test_inotify_event_creation(self):
+        event = InotfiyEvent(


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Copilot · 2025-11-21T08:06:36Z

checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py

+        self.assertEqual(watcher._mask2event(0x0000FFFF), "unknown")
+
+    @patch("checkbox_support.helpers.file_watcher.os.read")
+    @patch("checkbox_support.helpers.file_watcher.InotfiyEvent")


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Copilot · 2025-11-21T08:06:36Z

checkbox-support/checkbox_support/helpers/tests/test_file_watcher.py

+        self.assertEqual(events[0], mock_event_instance)
+
+    @patch("checkbox_support.helpers.file_watcher.os.read")
+    @patch("checkbox_support.helpers.file_watcher.InotfiyEvent")


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

Copilot · 2025-11-21T08:06:36Z

checkbox-support/checkbox_support/helpers/file_watcher.py

+                    else ""
+                )
+                event.append(
+                    InotfiyEvent(wd, self._mask2event(mask), cookie, name)


Corrected spelling of 'InotfiyEvent' to 'InotifyEvent'.

2. add WebGL conformance test script 3. add test cases and test plan 4. add readme

hanhsuan force-pushed the add_wegl_conformance_test branch 2 times, most recently from 8785d72 to 780b274 Compare September 22, 2025 12:06

hanhsuan marked this pull request as ready for review September 23, 2025 07:06

zongminl changed the title ~~Add wegl conformance test (New)~~ Add webgl conformance test (New) Sep 25, 2025

tomli380576 previously approved these changes Oct 14, 2025

View reviewed changes

providers/base/bin/webgl_conformance_test.py Show resolved Hide resolved

pieqq self-assigned this Nov 14, 2025

hanhsuan requested a review from Copilot November 21, 2025 08:05

Copilot AI reviewed Nov 21, 2025

View reviewed changes

hanhsuan dismissed tomli380576’s stale review via 4048fb5 November 23, 2025 02:28

hanhsuan added 2 commits November 23, 2025 11:01

1. add file_watcher in checkbox support

611ab41

2. add WebGL conformance test script 3. add test cases and test plan 4. add readme

Fix typo

bca26d4

hanhsuan force-pushed the add_wegl_conformance_test branch from 4048fb5 to bca26d4 Compare November 23, 2025 03:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add webgl conformance test (New) #2115

Add webgl conformance test (New) #2115

Uh oh!

hanhsuan commented Sep 17, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 17, 2025 •

edited

Loading

Uh oh!

tomli380576 left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Copilot AI Nov 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -0,0 +1,149 @@
		from checkbox_support.helpers.file_watcher import FileWatcher, InotfiyEvent

Add webgl conformance test (New) #2115

Are you sure you want to change the base?

Add webgl conformance test (New) #2115

Uh oh!

Conversation

hanhsuan commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Resolved issues

Documentation

Tests

Uh oh!

codecov bot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tomli380576 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hanhsuan commented Sep 17, 2025 •

edited

Loading

codecov bot commented Sep 17, 2025 •

edited

Loading