Add tool about visualiztion of checkbox test plans by rickwu666666 · Pull Request #137 · canonical/oem-qa-tools

rickwu666666 · 2026-04-24T05:39:50Z

Add a tool to parsing checkbox pxu file and display result on local webpage.

Copilot

Pull request overview

Adds a local FastAPI-based tool under Tools/test_plan_visualization/ to parse Checkbox .pxu units into a SQLite DB and provide a web UI for browsing jobs, exploring test plan trees, and comparing plans.

Changes:

Introduces a PXU parser + SQLite schema for jobs and test plans.
Adds FastAPI endpoints to query/filter jobs, render plan trees, show plan details, and compare plans.
Provides a self-contained run script, Dockerfile, and a single-page HTML/CSS/JS frontend.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
Tools/test_plan_visualization/run.sh	Bootstrap venv, clone/pull Checkbox repo, rebuild DB, start uvicorn
Tools/test_plan_visualization/requirements.txt	Python dependencies for the tool
Tools/test_plan_visualization/app/templates/index.html	Frontend UI for Jobs / Test Plans / Compare with modals
Tools/test_plan_visualization/app/parser.py	PXU parsing + DB rebuild logic (supports local providers priority)
Tools/test_plan_visualization/app/main.py	FastAPI app + API endpoints for jobs/options/plans/compare
Tools/test_plan_visualization/app/database.py	SQLAlchemy models + SQLite session management
Tools/test_plan_visualization/README.md	Usage and feature documentation
Tools/test_plan_visualization/Dockerfile	Containerization to run the tool with Docker

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    jobs = query.all()
+
+    if has_template_id is not None:
+        jobs = [
+            j
+            for j in jobs
+            if bool(
+                (json.loads(j.data) if j.data else {}).get("template-id", "")
+            )
+            == has_template_id
+        ]
+
+    if search:
+        sl = search.lower()
+
+        # helper: convert checkbox glob pattern to regex
+        def _matches(pattern: str, jid: str) -> bool:
+            try:
+                esc = _re.sub(r"\.(?!\*)", r"\\.", pattern)
+                esc = esc.replace(".*", "\x00")
+                esc = esc.replace(".", r"\.")
+                esc = esc.replace("\x00", ".*")
+                return bool(_re.match("^" + esc + "$", jid))
+            except _re.error:
+                return pattern == jid
+
+        # 1. Jobs whose own ID contains the search term
+        job_id_set = {j.job_id for j in jobs if sl in j.job_id.lower()}
+
+        # 2. Jobs that belong to test plans whose ID contains the search term
+        all_plans = db.query(TestPlan).all()
+        by_full_id = {p.full_id: p for p in all_plans}
+        by_bare_id = {p.plan_id: p for p in all_plans}
+
+        def _resolve(ref: str):
+            return by_full_id.get(ref) or by_bare_id.get(ref.split("::")[-1])
+
+        def _collect_patterns(plan, visited=None):
+            """Recursively collect include patterns from a plan
+            and all its nested_parts."""
+            if visited is None:
+                visited = set()
+            if plan.full_id in visited:
+                return set()
+            visited.add(plan.full_id)
+            patterns = set(json.loads(plan.include) if plan.include else [])
+            for child_ref in (
+                json.loads(plan.nested_part) if plan.nested_part else []
+            ):
+                child = _resolve(child_ref)
+                if child:
+                    patterns |= _collect_patterns(child, visited)
+            return patterns
+
+        matching_plans = [
+            p
+            for p in all_plans
+            if sl in p.full_id.lower() or sl in p.plan_id.lower()
+        ]
+
+        plan_job_id_set = set()
+        for plan in matching_plans:
+            patterns = _collect_patterns(plan)
+            for job in jobs:
+                bare_job = job.job_id.split("::")[-1]


+#!/bin/bash
+set -e
+
+REPO_URL="https://github.com/canonical/checkbox.git"
+REPO_DIR="checkbox_repo"
+VENV_DIR=".venv"
+


+            currentJobs.forEach((job, i) => {
+                const tr = document.createElement('tr');
+                const envTags = job.environ.map(e => `<span class="tag env">${e}</span>`).join('');
+                const manTags = job.manifest.map(m => `<span class="tag man">${m}</span>`).join('');
+                const provider = job.provider.split('::').pop();
+                const category = job.category.split('::').pop();
+                const highlight = q
+                    ? job.id.replace(new RegExp(`(${q.replace(/[.*+?^${}()|[\]\\]/g,'\\$&')})`, 'gi'),
+                        '<mark style="background:#fff3cd;border-radius:2px">$1</mark>')
+                    : job.id;
+                tr.innerHTML = `
+                    <td style="color:#bbb;font-size:.73rem">${i + 1}</td>
+                    <td><span class="job-id">${highlight}</span></td>
+                    <td><span class="tmpl-id">${job.template_id || ''}</span></td>
+                    <td style="font-size:.76rem;color:#667">${provider}</td>
+                    <td style="font-size:.76rem;color:#667">${category}</td>
+                    <td>${envTags}</td>
+                    <td>${manTags}</td>
+                    <td style="font-size:.8rem">${job.summary || ''}</td>
+                    <td><button class="btn-details" onclick="showDetails(${i})">Details</button></td>
+                `;


+app.mount("/static", StaticFiles(directory="app/static"), name="static")
+templates = Jinja2Templates(directory="app/templates")


+
+@app.get("/", response_class=HTMLResponse)
+async def read_items(request: Request):
+    return templates.TemplateResponse(request=request, name="index.html")


+    if environ:
+        query = query.filter(Job.environ.contains(environ))
+
+    if manifest:
+        query = query.filter(Job.manifest.contains(manifest))
+


+        # Apply this plan's own excludes to the full accumulated set
+        if own_excludes:
+            result = {
+                jid
+                for jid in result
+                if not any(
+                    _matches(ep.split("::")[-1], jid.split("::")[-1])
+                    or _matches(ep.split("::")[-1], jid)
+                    for ep in own_excludes


+
+if [ "$NEEDS_DB_UPDATE" = true ]; then
+    echo "Updating database..."
+    python3 -c "from app.parser import update_db; update_db('checkbox_repo', 'providers')"


+            let attrsHtml = '<p class="section-title" style="margin-top:20px">Job Attributes</p><div class="attr-grid">';
+            for (const k of keys) {
+                const v = String(attrs[k]).trim();
+                attrsHtml += `<div class="attr-key">${k}</div><div class="attr-val"><pre>${v}</pre></div>`;
+            }


+a local search engine for job units and test plans. It supports two views:
+
+- **Jobs View** — browse and filter individual job units
+- **Test Plans View** — search test plans and explore their nested structure
+  down to included jobs
+
+## Features
+
+- **Two-View UI**: Toggle between Jobs and Test Plans views from the header.


hanhsuan · 2026-04-29T07:53:20Z

Checkbox is going to migrate pxu to yaml. Therefore, it would be better to do it for pxu and yaml at the same time. However, it could be easier to use this.

rickwu666666 · 2026-04-29T08:23:33Z

Checkbox is going to migrate pxu to yaml. Therefore, it would be better to do it for pxu and yaml at the same time. However, it could be easier to use this.

To adopt both pxu and yaml is a good point. However, this tool intend to not involve any checkbox function in case we have to touch the code in the future.

hanhsuan · 2026-04-29T08:39:11Z

However, this tool intend to not involve any checkbox function in case we have to touch the code in the future.

I don't understand what this means. If you don't what to depend on any checkbox feature, how do you know the list of test cases is same as checkbox ?

rickwu666666 · 2026-04-29T13:03:07Z

However, this tool intend to not involve any checkbox function in case we have to touch the code in the future.

I don't understand what this means. If you don't what to depend on any checkbox feature, how do you know the list of test cases is same as checkbox ?

I want to move away from the checkbox function to prevent data errors caused by unexpected changes of checkbox. If the parser is implemented correctly, it should match the checkbox output since the job/plan definitions are clear.
But you're right—the current parser is still a bit rough and the plan comparison logic needs optimization. I'm switching this PR to draft and will reopen it when the implementation is more solid.

… than query from db.

Add tool about visualiztion of checkbox test plans

489729a

rickwu666666 marked this pull request as draft April 24, 2026 06:42

rickwu666666 force-pushed the checkbox-test-plan-visualization branch 2 times, most recently from 12d8072 to 039ea07 Compare April 27, 2026 01:33

Fix format

a1974f2

rickwu666666 force-pushed the checkbox-test-plan-visualization branch from 039ea07 to a1974f2 Compare April 27, 2026 01:46

rickwu666666 marked this pull request as ready for review April 27, 2026 01:50

rickwu666666 requested a review from a team April 27, 2026 01:50

rickwu666666 added 2 commits April 28, 2026 19:12

Add providers folder for sideload provider

6b6af22

Fix markdown

22fdb39

seankingyang requested review from Copilot and removed request for a team April 28, 2026 11:23

Copilot started reviewing on behalf of seankingyang April 28, 2026 11:24 View session

seankingyang self-requested a review April 28, 2026 11:27

Copilot AI reviewed Apr 28, 2026

View reviewed changes

rickwu666666 added 2 commits April 28, 2026 19:44

Update the result of compare page to show the difference of two plans

2baff91

Fix wrong exclude job listed in compare page

8978c1a

rickwu666666 marked this pull request as draft April 29, 2026 13:03

Use checkbox-cli expand for compare page to make it much more quicker…

79e2e40

… than query from db.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add tool about visualiztion of checkbox test plans#137

Add tool about visualiztion of checkbox test plans#137
rickwu666666 wants to merge 7 commits intomainfrom
checkbox-test-plan-visualization

rickwu666666 commented Apr 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

hanhsuan commented Apr 29, 2026

Uh oh!

rickwu666666 commented Apr 29, 2026

Uh oh!

hanhsuan commented Apr 29, 2026

Uh oh!

rickwu666666 commented Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		app.mount("/static", StaticFiles(directory="app/static"), name="static")
		templates = Jinja2Templates(directory="app/templates")

Conversation

rickwu666666 commented Apr 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

hanhsuan commented Apr 29, 2026

Uh oh!

rickwu666666 commented Apr 29, 2026

Uh oh!

hanhsuan commented Apr 29, 2026

Uh oh!

rickwu666666 commented Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants