Skip to content

Commit a1974f2

Browse files
committed
Fix format
1 parent 489729a commit a1974f2

4 files changed

Lines changed: 337 additions & 186 deletions

File tree

Tools/test_plan_visualization/README.md

Lines changed: 67 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,42 +4,61 @@ A web application to inspect, filter, and explore Checkbox jobs and test plans.
44

55
## Overview
66

7-
This application parses the [Checkbox](https://github.com/canonical/checkbox) repository and provides a local search engine for job units and test plans. It supports two views:
7+
This application parses the
8+
[Checkbox](https://github.com/canonical/checkbox) repository and provides
9+
a local search engine for job units and test plans. It supports two views:
810

911
- **Jobs View** — browse and filter individual job units
10-
- **Test Plans View** — search test plans and explore their nested structure down to included jobs
12+
- **Test Plans View** — search test plans and explore their nested structure
13+
down to included jobs
1114

1215
## Features
1316

1417
- **Two-View UI**: Toggle between Jobs and Test Plans views from the header.
15-
- **Automated Parsing**: Scans `.pxu` files and extracts both job units (including legacy `plugin:`-style entries) and test plan units.
16-
- **Job Filters**: Filter by Provider, Category, Environ, Manifest Keys, and Template ID presence.
17-
- **Search**: Search by job ID, test plan ID, or plan name — searches across both jobs and the test plans they belong to.
18-
- **Test Plan Tree**: In the Test Plans view, expand plans to see nested sub-plans and directly included jobs.
19-
- **Exclude Support**: Jobs excluded by a test plan's `exclude:` field are shown with a strikethrough and an EXCLUDED badge — they remain visible but clearly marked.
20-
- **Job Details Modal**: Click Details on any job to see its full attributes and the complete test plan hierarchy it belongs to.
21-
- **Plan Details Modal**: Click Details on any test plan card to see all raw plan attributes, include/exclude patterns, and nested parts.
22-
- **Dynamic Filters**: Dropdown options update dynamically based on current selections.
23-
- **Provider Resolution**: Automatically resolves provider namespaces from `manage.py`.
24-
- **Compare Plans**: Switch to the Compare view, enter two plan IDs, and see a three-column diff — jobs only in Plan 1, jobs in both, and jobs only in Plan 2 (excludes are applied before comparing).
18+
- **Automated Parsing**: Scans `.pxu` files and extracts both job units
19+
(including legacy `plugin:`-style entries) and test plan units.
20+
- **Job Filters**: Filter by Provider, Category, Environ, Manifest Keys,
21+
and Template ID presence.
22+
- **Search**: Search by job ID, test plan ID, or plan name — searches across
23+
both jobs and the test plans they belong to.
24+
- **Test Plan Tree**: In the Test Plans view, expand plans to see nested
25+
sub-plans and directly included jobs.
26+
- **Exclude Support**: Jobs excluded by a test plan's `exclude:` field are
27+
shown with a strikethrough and an EXCLUDED badge — they remain visible
28+
but clearly marked.
29+
- **Job Details Modal**: Click Details on any job to see its full attributes
30+
and the complete test plan hierarchy it belongs to.
31+
- **Plan Details Modal**: Click Details on any test plan card to see all raw
32+
plan attributes, include/exclude patterns, and nested parts.
33+
- **Dynamic Filters**: Dropdown options update dynamically based on
34+
current selections.
35+
- **Provider Resolution**: Automatically resolves provider namespaces
36+
from `manage.py`.
37+
- **Compare Plans**: Switch to the Compare view, enter two plan IDs, and see
38+
a three-column diff — jobs only in Plan 1, jobs in both, and jobs only in
39+
Plan 2 (excludes are applied before comparing).
2540

2641
## Getting Started
2742

2843
### Option A: Run Locally
2944

3045
**Prerequisites:** Python 3.10+, `git`
3146

32-
1. **Run the startup script** — it automatically creates a virtual environment, installs dependencies, clones/updates the checkbox repo, builds the database, and starts the server:
47+
1. **Run the startup script** — it automatically creates a virtual
48+
environment, installs dependencies, clones/updates the checkbox repo,
49+
builds the database, and starts the server:
50+
3351
```bash
3452
./run.sh
3553
```
3654

3755
> To restart (e.g. after a code change):
56+
>
3857
> ```bash
3958
> pkill -f "uvicorn app.main:app"; ./run.sh
4059
> ```
4160

42-
2. **Access the Web Interface**:
61+
2. **Access the Web Interface**:
4362
Open your browser to [http://localhost:8888](http://localhost:8888).
4463

4564
---
@@ -48,26 +67,34 @@ This application parses the [Checkbox](https://github.com/canonical/checkbox) re
4867

4968
**Prerequisites:** [Docker](https://docs.docker.com/get-docker/)
5069

51-
1. **Build the Docker image**:
70+
1. **Build the Docker image**:
71+
5272
```bash
5373
sudo docker build -t checkbox-job-db .
5474
```
5575

56-
2. **Run the container** (the startup script runs automatically inside the container):
76+
2. **Run the container** (the startup script runs automatically
77+
inside the container):
78+
5779
```bash
5880
sudo docker run -p 8888:8888 checkbox-job-db
5981
```
6082

61-
> If you get a "port already allocated" error, stop any existing container first:
83+
> If you get a "port already allocated" error, stop any existing
84+
> container first:
85+
>
6286
> ```bash
6387
> sudo docker stop $(sudo docker ps -q --filter publish=8888)
6488
> ```
65-
> If the port still appears stuck with no process using it, restart the Docker daemon:
89+
>
90+
> If the port still appears stuck with no process using it, restart
91+
> the Docker daemon:
92+
>
6693
> ```bash
6794
> sudo systemctl restart docker
6895
> ```
6996

70-
3. **Access the Web Interface**:
97+
3. **Access the Web Interface**:
7198
Open your browser to [http://localhost:8888](http://localhost:8888).
7299

73100
## Project Structure
@@ -82,52 +109,67 @@ This application parses the [Checkbox](https://github.com/canonical/checkbox) re
82109

83110
## API Endpoints
84111

112+
<!-- markdownlint-disable MD013 -->
85113
| Endpoint | Description |
86-
|---|---|
114+
| --- | --- |
87115
| `GET /api/jobs` | List jobs with optional filters: `provider`, `category`, `environ`, `manifest`, `has_template_id`, `search` |
88116
| `GET /api/options` | Get available filter values for the current filter selection |
89117
| `GET /api/testplans?job_id=` | Get the full test plan ancestry for a given job ID |
90118
| `GET /api/plan-tree?search=` | Search test plans and return their full nested tree with included jobs (excluded jobs marked) |
91119
| `GET /api/plan-details?plan_id=` | Get all attributes, include/exclude patterns, and nested parts for a single test plan |
92120
| `GET /api/compare-plans?plan1=&plan2=` | Compare effective job sets of two test plans (excludes applied), returning only-in-1, in-both, only-in-2 |
121+
<!-- markdownlint-enable MD013 -->
93122

94123
## Troubleshooting: Port 8888
95124

96125
**Check what is using port 8888:**
126+
97127
```bash
98128
ss -tlnp sport = :8888 # shows process if owned by current user
99-
sudo ss -tlnp sport = :8888 # shows process for all users (including Docker/root)
129+
sudo ss -tlnp sport = :8888 # shows process for all users
130+
# (including Docker/root)
100131
```
101132

102133
**Kill local uvicorn process:**
134+
103135
```bash
104136
pkill -f "uvicorn app.main:app"
105137
```
106138

107139
**Kill via port (requires root for Docker-owned processes):**
140+
108141
```bash
109142
sudo fuser -k 8888/tcp
110143
```
111144

112145
**Stop Docker container holding the port:**
146+
113147
```bash
114148
sudo docker stop $(sudo docker ps -q --filter publish=8888)
115149
```
116150

117151
**List running containers and their ports:**
152+
118153
```bash
119154
sudo docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}"
120155
```
121156

122157
**If port appears stuck with no process using it (stale Docker state):**
158+
123159
```bash
124160
sudo systemctl restart docker
125161
```
126162

127163
## Notes
128164

129-
- The database is rebuilt every time the server starts (both locally and in Docker).
165+
- The database is rebuilt every time the server starts
166+
(both locally and in Docker).
130167
- Both `unit: job` and legacy `plugin:`-style job blocks are parsed.
131-
- Test plan nested hierarchy is traversed recursively; cycle detection is built in.
132-
- The `exclude:` field in test plans is parsed and stored. Excluded jobs are shown with a strikethrough in the plan tree and are removed from the effective job set when comparing plans.
133-
- Environment variables referenced in a job's `command:` field (e.g. `$SERIAL_PORTS_STATIC`) are automatically added to the Environ filter even if no explicit `environ:` field is declared.
168+
- Test plan nested hierarchy is traversed recursively;
169+
cycle detection is built in.
170+
- The `exclude:` field in test plans is parsed and stored. Excluded jobs
171+
are shown with a strikethrough in the plan tree and are removed from
172+
the effective job set when comparing plans.
173+
- Environment variables referenced in a job's `command:` field
174+
(e.g. `$SERIAL_PORTS_STATIC`) are automatically added to the Environ
175+
filter even if no explicit `environ:` field is declared.

Tools/test_plan_visualization/app/database.py

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
from sqlalchemy import create_engine, Column, Integer, String, Text, ForeignKey
1+
from sqlalchemy import create_engine, Column, Integer, String, Text
22
from sqlalchemy.orm import declarative_base
33
from sqlalchemy.orm import sessionmaker
44

@@ -11,33 +11,40 @@
1111

1212
Base = declarative_base()
1313

14+
1415
class Job(Base):
1516
__tablename__ = "jobs"
1617

1718
id = Column(Integer, primary_key=True, index=True)
1819
job_id = Column(String, index=True)
1920
provider = Column(String, index=True)
2021
category_id = Column(String, index=True)
21-
environ = Column(Text) # Stored as JSON string or comma separated
22-
manifest = Column(Text) # Stored as JSON string or comma separated keywords from `requires`
22+
environ = Column(Text) # Stored as JSON string or comma separated
23+
manifest = Column(
24+
Text
25+
) # Stored as JSON string or comma separated keywords from `requires`
2326
command = Column(Text)
2427
summary = Column(Text)
2528
description = Column(Text)
26-
unit_type = Column(String) # 'job', 'test plan', etc.
27-
data = Column(Text) # JSON string of all attributes
29+
unit_type = Column(String) # 'job', 'test plan', etc.
30+
data = Column(Text) # JSON string of all attributes
31+
2832

2933
class TestPlan(Base):
3034
__tablename__ = "test_plans"
3135

3236
id = Column(Integer, primary_key=True, index=True)
33-
plan_id = Column(String, index=True) # e.g. bluetooth-cert-automated
34-
full_id = Column(String, index=True) # with namespace, e.g. com.canonical.certification::bluetooth-cert-automated
37+
plan_id = Column(String, index=True) # e.g. bluetooth-cert-automated
38+
full_id = Column(
39+
String, index=True
40+
) # e.g. com.canonical.certification::bluetooth-cert-automated
3541
provider = Column(String, index=True)
3642
name = Column(Text)
37-
include = Column(Text) # JSON list of raw include patterns
38-
exclude = Column(Text) # JSON list of raw exclude patterns
43+
include = Column(Text) # JSON list of raw include patterns
44+
exclude = Column(Text) # JSON list of raw exclude patterns
3945
nested_part = Column(Text) # JSON list of nested test plan IDs
40-
data = Column(Text) # JSON of all raw attributes
46+
data = Column(Text) # JSON of all raw attributes
47+
4148

4249
def get_db():
4350
db = SessionLocal()

0 commit comments

Comments
 (0)