Add Crucible (Mesa CTS) Testing (New)#2461
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2461 +/- ##
==========================================
+ Coverage 58.86% 59.07% +0.20%
==========================================
Files 476 479 +3
Lines 48007 48229 +222
Branches 8570 8623 +53
==========================================
+ Hits 28260 28490 +230
+ Misses 18855 18840 -15
- Partials 892 899 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
At time of writing, the Vulkan Classic test plan will fail because this job is still running (meaning I cannot promote the snap with the needed changes): This one has been a bit trickier to integrate than the past two, so I'm definitely looking for some feedback where folks see a need for additional polishing. I've been trying to leave our options open for non-amd64 hardware, but the crucible snap does not have a build for any other arch at the moment. |
|
The vulkan snap has been updated, so the vulkan classic test plan will succeed. |
| #!/usr/bin/env python3 | ||
| # This file is part of Checkbox. | ||
| # | ||
| # Copyright 2025 Canonical Ltd. |
There was a problem hiding this comment.
Minor: lol
| # Copyright 2025 Canonical Ltd. | |
| # Copyright 2026 Canonical Ltd. |
| # NODEVICE_SELECT disables VK_LAYER_MESA_device_select — the layer | ||
| # requires GLIBC_ABI_GNU2_TLS, which the snap's core24 glibc lacks. |
There was a problem hiding this comment.
Is this a TODO to remove this on core26?
There was a problem hiding this comment.
Ideally, yes. The device selection via VK_LAYER_MESA_device_select fails if the snap has a different glibc version than the host, which I suspect could continue to be an issue in the future. For instance, if we need to test on 26.10 with core26, the same issue should come back
There was a problem hiding this comment.
if you want to import this, it must move to the checkbox_support module. Probably under helpers.
| except (FileNotFoundError, OSError, subprocess.CalledProcessError): | ||
| return None |
There was a problem hiding this comment.
No, don't hide exceptions, rise them. You can group them all into a single one if you want.
There was a problem hiding this comment.
Done, converted everything to raise VulkanDetectionError
There was a problem hiding this comment.
Ok I don't want to write the same comment on the whole file. In python we don't return None as a special value to say error. Error = Exception. I don't like exceptions but that is how the language works.
| except OSError: | ||
| continue | ||
| except OSError: | ||
| pass | ||
| return None |
There was a problem hiding this comment.
? I'm sure we can do better than this, 2 nested os errors? this is really hard to test/reason about
There was a problem hiding this comment.
Refactored to remove this pattern
| result.append(path) | ||
| except (OSError, ValueError): | ||
| result.append(path) |
There was a problem hiding this comment.
Why would we add the path erroring to result? At least comment on this action if you have to do it
There was a problem hiding this comment.
A holdover from an earlier experiment when the correct path was not being added in the try block. Thanks for catching, I've changed it
| except OSError: | ||
| pass |
There was a problem hiding this comment.
Same as above, are we sure we need two nested os errors? Remember suppress exists and makes your life easier AND raising exceptions is actually idiomatic if you want to say somthing went wrong
| estimated_duration: 1s | ||
| command: | ||
| if vulkaninfo --summary 2>/dev/null | grep -q "PHYSICAL_DEVICE_TYPE_.*_GPU"; then | ||
| if env -u DISPLAY vulkan-cts.vulkaninfo --summary 2>&1 | grep "PHYSICAL_DEVICE_TYPE_.*_GPU"; then |
There was a problem hiding this comment.
Comment why you env -u that variable. Also, weird to do this in bash considering we did everything else in python
There was a problem hiding this comment.
Added a comment to explain that this is for a more headless experience
2 participants
Description
Crucible is a test suite for Mesa conformance.
Some notes in this PR:
Resolved issues
Mesa is already tested by the Vulkan CTS testing, but Crucible is the testing developed specifically for Mesa, so it is a great tool to make sure that it is integrated properly from the perspective of the upstream developers.
Documentation
https://gitlab.freedesktop.org/mesa/crucible
Tests