Skip to content

[quality] test(ujust): add BATS unit tests for confirm/verify recipes - #1309

Open
kubestellar-hive[bot] wants to merge 217 commits into
mainfrom
quality/test-ujust-confirm-verify
Open

[quality] test(ujust): add BATS unit tests for confirm/verify recipes#1309
kubestellar-hive[bot] wants to merge 217 commits into
mainfrom
quality/test-ujust-confirm-verify

Conversation

@kubestellar-hive

@kubestellar-hive kubestellar-hive Bot commented Aug 7, 2026

Copy link
Copy Markdown

Test Improvement

Adds tests/unit/confirm-verify_test.bats with 23 BATS tests covering the security-hardened ujust confirm and ujust verify recipes introduced in PR #1136.

What is tested

Cancel/abort gate (security-critical: ensures nothing is posted without user consent)

  • gum confirm decline exits 0 with "Cancelled." and makes no gh issue comment call

Unauthenticated fallback

  • GitHub URL printed, no comment posted
  • Clipboard fallback (wl-copy) when available

System fingerprint collection

  • Image, digest, kernel, arch, Device ID all present in comment body
  • bootc failure falls back gracefully to "unknown" values without crashing

Failed units

  • systemctl failures included in confirm comment; absent when none

Verify verdicts (all 4 paths)

  • fixed, still-broken, different, cannot-confirm result strings correct

Verify steps

  • Displayed when present in issue body; generic message when absent

Journal consent + redaction

  • journalctl invoked for still-broken/different verdicts when user consents
  • IPs and home paths redacted from journal output before inclusion

Also adds test-unit target to Justfile: just test-unit runs bats tests/unit/.

Related Issue

Closes #1179

Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.59

castrojo and others added 30 commits June 29, 2026 22:13
@castrojo
docs: clean house — delete stale docs, align all docs with OCI-native…
7b88545 
… daily factory (#1090)

Delete banned changelog/planning files:
- docs/factory/IMPROVEMENTS.md: banned changelog file (per skill-improvement.md)
- docs/epics.md: alpha-era planning doc with issues 200-548; GitHub board is source of truth

Fix factual errors in CI/promotion docs:
- docs/ci.md: rewrite stable promotion section; remove deleted promote-testing-to-main.yml
  refs and squash-PR flow; fix schedule note (daily 13:00 UTC); fix testing branch attribution
- AGENTS.md: fix stale promotion PR reference -> execute-release.yml description; fix
  weekly squash PR -> daily automated; fix :testing/:stable cadence description
- docs/skills/overview.md: fix image stream table (testing on testing branch, not main;
  daily not weekly promotion)
- docs/skills/quickstart.md: branch from upstream/testing; remove outdated human-approval
  step 5 (weekly-testing-promotion.yml + 2 approvals)
- docs/skills/aarch64.md: rewrite triggers (workflow_run + push + dispatch); remove
  deleted cache-warm.yml section; fix hard rules to remove deleted workflow refs
- docs/skills/workflow-map.md: move boot-check to publish.yml (where it actually is);
  remove it from execute-release.yml; add push trigger to build-aarch64.yml entry
- docs/skills/release-promotion.md: fix boot-check placement in promotion map and core process
- docs/skills/ci-tooling.md: replace sections 12+13 with DELETED pattern notes; remove
  stale red flag about sync-main-to-testing
- docs/skills/ci.md: add Lessons Learned banner noting pre-2026-06-23 entries may reference
  deleted workflows; fix execute-release quick reference (no boot-check)

Fix other doc errors:
- README.md: fix image stream table (testing branch, daily not weekly); remove duplicate
  Help section; fix agent-ready queue label (status/queued not queue/agent-ready)
- CONTRIBUTING.md: fix directory table (patches/+files/ instead of nonexistent junctions/
  +system_files/); fix bst install method (just bst, not pip); fix just check -> validate+lint
- docs/build.md: fix force-push note for release bookmark model

Add ci.md to routing tables:
- docs/skills/README.md: add ci.md as load-first for CI failures
- docs/SKILL.md: add ci.md entry to Task -> Skill table

Closes 0 issues. This is a documentation clean-house aligned with the OCI-native
automated factory that shipped 2026-06-23.

Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
docs(skills): record statuses:write startup_failure pattern (2026-06-…
557355b 
…23) (#1071)

When projectbluefin/actions@v1 adds a new job-level permission to the
reusable promote workflow, the caller must grant it at the top level.
Missing permissions cause startup_failure with no log output.

Symptom confirmed: 3 consecutive startup_failures on promote dispatch
after actions@v1 was updated to post validate commit status.
Fix: add statuses:write to promote-testing-to-main.yml permissions.

Assisted-by: Claude Sonnet 4.6 via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bketelsen @castrojo
feat(bluefin): add Incus native library elements (1/2) (#1125)
ed7541a 
* feat(bluefin): add Incus native library elements

Add the from-source BuildStream elements for Incus's native dependency
stack. None of these exist in the freedesktop-sdk / gnome-build-meta
junctions, so they are authored here:

- libuv (cmake)
- raft (autotools) — cowsql's Raft consensus
- cowsql (autotools) — embedded replicated SQLite (dqlite fork)
- liblxc (meson) — system-container runtime
- lxcfs (meson) — cgroup-aware /proc for containers

Consumed by bluefin/incus.bst in the follow-up PR. Validated by a full
oci/bluefin.bst image build.

Part of #1124

Assisted-by: Claude Opus 4.8 (1M context)

* fix(incus): remove obsolete cowsql raft configure flag

Assisted-by: Claude Sonnet 4.6 via GitHub Copilot

---------

Co-authored-by: Jorge O. Castro <jorge.castro@gmail.com>
@bketelsen @castrojo
feat(bluefin): add Incus daemon, systemd integration, and ujust recip…
6d9c8b3 
…es (#1126)

Wire Incus into the image on top of the native library elements:

- bluefin/incus.bst: build incusd + incus + helpers from the vendored
  release tarball (CGO, -tags libsqlite3, static agent/migrate). Pulls
  QEMU/OVMF/virtiofsd/swtpm/dnsmasq/nftables/storage tooling from the
  junctions as runtime deps.
- deps.bst: add bluefin/incus.bst to the image manifest.
- files/incus/: systemd units (incus.socket/service, incus-startup,
  incus-user), preset (enables sockets + lxcfs), sysusers.d
  (incus-admin/incus groups), tmpfiles.d (/run/incus + state dirs),
  modules-load.d (vhost_vsock/vhost_net/nbd/br_netfilter),
  /etc/default/incus (INCUS_EDK2_PATH), and /etc/subuid+subgid.
- system.just: `ujust incus-init` and `ujust incus-status`.

Includes overlap-whitelist for /etc/subuid and /etc/subgid.
Builds green end-to-end (oci/bluefin.bst). Runtime boot/launch
validation tracked in the issue.

Closes #1124

Assisted-by: Claude Opus 4.8 via GitHub Copilot
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot

Co-authored-by: Jorge O. Castro <jorge.castro@gmail.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to 16b0581 (#1127)
f120eb8 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@mergeraptor
chore(deps): update github/codeql-action action to v4.36.3 (#1135)
49d6cd9 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
fix(ci): bust stale oci/bluefin cache key
3dca4ea 
Add a no-op command in oci/bluefin.bst so BuildStream generates a new artifact key instead of repeatedly pulling the stale remote key that fails with blob download code 13.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to c93ccc0 (#1137)
d287e85 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
fix(ci): align scorecard push trigger with default branch (#1138)
a3eef87 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
docs(skills): document cache-busting of corrupt remote CAS keys
f4d8d94 
Assisted-by: Gemini 3.5 Flash via GitHub Copilot
@castrojo
Merge remote-tracking branch 'upstream/testing' into testing
bbd0159
@castrojo
fix(ci): raise testing build timeout budget
fd55f45 
Increase BuildStream time limits in build.yml (job 420m, step 390m) so testing builds can finish after cold-ish GNOME delta rebuilds instead of dying at the previous 330m step timeout.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
fix(oci): force new bluefin layer blob after CAS mismatch
9001b98 
Revert timeout-only tuning in build.yml and replace the no-op oci/bluefin cache-bust with a deterministic layer-content marker file so the produced OCI layer blob digest changes.

Assisted-by: GPT-5.3-Codex via GitHub Copilot
@castrojo
docs(skills): add concrete CAS digest mismatch fix (commit 9001b98)
a8424cc 
Documented the deterministic layer content workaround with specific example
from commit 9001b98 that replaced no-op 'true' with cas-epoch marker file.
This helps future agents understand why the fix bypasses poisoned blobs.
@castrojo
fix(oci): force new bluefin layer blob for CAS mismatch
a97912f 
Write a deterministic marker file into the bluefin OCI layer so BuildStream
gets a new blob digest instead of reusing a poisoned remote object. The
previous no-op cache-bust did not change the produced layer bytes.

Assisted-by: GitHub Copilot
@castrojo
fix(ci): raise dakota build timeout budget
f0320c9 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(oci): revert timeout workaround and keep CAS-layer fix
fca33c3 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(oci): move digest marker into layer compose element
d11a35a 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@mergeraptor
chore(deps): update taiki-e/install-action digest to 4684b84 (#1139)
89692e2 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
ci(build): align build step timeout with job budget
b26b399 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
ci(build): raise remote BST timeout budget
304c934 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): enable remote execution in build workflow
e60c3ac 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): fail fast when remote execution config is missing
650a5fc 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
docs(skills): capture remote-build evidence checklist
9a31980 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): reduce BuildStream parallelism for GCC bootstrap
bbb09f0 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): reduce BST max jobs for gcc bootstrap
9af4301 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): reduce BST max jobs for bootstrap toolchains
979bef4 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@castrojo
fix(ci): serialize BuildStream builders for bootstrap
bf21834 
Assisted-by: Claude Sonnet 4.6 via GitHub Copilot
@mergeraptor
chore(deps): update taiki-e/install-action digest to 5041467 (#1141)
f470fa4 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
fix(ci): override freedesktop-sdk build flags to -O1 to fix GCC 15 ICE
f860fda 
Assisted-by: Gemini 3.5 Flash via GitHub Copilot
mergeraptor Bot and others added 27 commits August 1, 2026 15:11
@mergeraptor
chore(deps): update taiki-e/install-action digest to 1beb33e (#1261)
2f93a9e 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@ahmedadan
fix(zswap): disable zram via integration command, overlap lost (#1262)
53952c7
@castrojo
fix(ci): delete .github/workflows/lab-check.yml (#1260)
52412d0 
YAGNI
@castrojo
ci: delete skill-drift.yml (calls a deprecated no-op stub) (#1263)
5310c31
@castrojo
docs: drop stale skill-drift references (the check no longer exists)
2f950d7 
Assisted-by: Claude Opus 5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to 67729d5 (#1265)
5ca1d9b 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
feat(bluefin): add ydotool input automation (#1258)
a182aba 
Build ydotool v1.0.4 from source, ship its opt-in user daemon, and provide uinput module and uaccess rules for Wayland input injection.\n\nCloses #1217\n\nAssisted-by: GPT-5.6 Luna via GitHub Copilot\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: castrojo <castrojo@users.noreply.github.com>
@ahmedadan
fix(bluefin): support ydotool on CMake 4 (#1267)
599e2b4 
CMake 4 removed compatibility with policy versions older than 3.5, while ydotool v1.0.4 still declares 3.4. Set the documented package-local policy floor so the pinned release configures and builds.

Assisted-by: GPT-5.6 Sol via pi
@mergeraptor @github-actions
chore(deps): update common: v2026.07-63 -> v2026.08-6 (#1266)
eb96a44 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@mergeraptor
chore(deps): update github/codeql-action action to v4.37.5 (#1272)
220b83e 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@ahmedadan
feat(nvidia): update to 610.57.04 and ship its missing data files (#1277
ed2bea8 
)
@ahmedadan
fix(gaming): ship 32-bit GL/Vulkan drivers and re-enable Steam UI GPU (…
096b351 
…#1278)
@ahmedadan
fix(gaming): pin Steam .deb to versioned pool URL; CDN swaps content (#…
e051009 
…1279)
@mergeraptor
chore(deps): update github/codeql-action action to v4.37.6 (#1282)
5fa20ce 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to cb33e69 (#1284)
93c35ad 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@castrojo
fix(audio): disable PC-speaker WirePlumber sink (#1257)
a5e9152 
Hide the snd_pcsp ALSA card from desktop audio policy while keeping the kernel driver available for console beeps. This prevents the 37 kHz mono fallback from becoming the default sink after suspend and resume.

Fixes #1248

Assisted-by: GPT-5.6 Luna via GitHub Copilot

Co-authored-by: castrojo <castrojo@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ahmedadan
fix(audio): unfold pcsp install command; YAML folding emptied artifact (
b32e6fe 
#1297)
@castrojo
fix(elements): stop using /dev/stdin for inline file writes (#1298)
5e30bde 
Under BuildBarn/BuildBox remote execution the action sandbox chroots into
the input root, which has no /dev/stdin. Every element writing an inline
config file via `install -Dm644 /dev/stdin ... <<'EOF'` therefore fails
with:

    install: cannot stat '/dev/stdin': No such file or directory

This blocked the distributed Dakota build at bluefin/fprintd-system-auth
and bluefin/migrate-var-home-passwd.

docs/skills/buildstream.md already documents the correct two-step
pattern; it had simply never been applied. Use it at all 9 element sites
and in the four packaging docs that still taught the broken form:

    install -Dm644 /dev/null "target"
    cat > "target" <<'EOF'

Parent-directory creation and file mode are preserved by `install -D`,
so behaviour is unchanged for local builds.

Co-authored-by: castrojo <castrojo@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to 91ddec7 (#1295)
26402b3 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@mergeraptor
chore(deps): update taiki-e/install-action digest to 6c6fd71 (#1300)
2a0fb4f 
Co-authored-by: mergeraptor[bot] <267480593+mergeraptor[bot]@users.noreply.github.com>
@mergeraptor @github-actions
chore(deps): update uutils-coreutils: 0.9.0-0 -> 0.10.0-0 (#1293)
1bc0e86 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@mergeraptor @github-actions
chore(deps): update bootc: v1.16.6-0 -> v1.16.7-0 (#1292)
2c9eb26 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@mergeraptor @github-actions
chore(deps): update tailscale 1.98.10 -> 1.102.2 (#1280)
6df76f0 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@mergeraptor @github-actions
chore(deps): update jetbrains-mono-nerd-font v3.4.0 -> v3.5.0 (#1270)
c24d41f 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@mergeraptor @github-actions
chore(deps): update common: v2026.08-6 -> v2026.08-11 (#1269)
aadd447 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@castrojo
docs(skills): record /dev/stdin regression and why CI cannot catch it (…
ecc983e 
…#1303)

The nine sites fixed in #1298 reintroduced the /dev/stdin pattern after
the 2026-07-25 lesson was written. GHA BuildBox mounts /proc via
bubblewrap so CI stays green; only the lab BuildBarn grid (bb_runner
chroot, no /proc) fails. Record the asymmetry and the review check.

Assisted-by: Kimi K3 via GitHub Copilot
quality agent
test(ujust): add BATS unit tests for confirm/verify recipes
8e7cc44 
Add tests/unit/confirm-verify_test.bats covering the security-hardened
confirm and verify ujust recipes introduced in PR #1136.

23 test cases covering:
- Cancel/abort gate: gum decline exits 0 without posting (both recipes)
- Unauthenticated fallback: URL printed, no gh issue comment called
- Clipboard fallback: wl-copy invoked when present and unauthenticated
- System fingerprint: image, digest, kernel, arch, Device ID in comment
- bootc failure: graceful fallback to 'unknown' values, still posts
- Failed units: systemctl failures included in confirm comment
- Verify verdicts: fixed, still-broken, different, cannot-confirm
- Verify steps: displayed when present in issue body, generic message when absent
- Journal consent: journalctl invoked for still-broken/different verdicts
- Journal redaction: IPs and home paths removed before inclusion
- Footer: ujust confirm/verify invocation line present in comment

Add test-unit Justfile target: bats tests/unit/

Closes #1179

Signed-off-by: quality agent <quality-agent@hive.local>
@kubestellar-hive
kubestellar-hive Bot requested a review from a team as a code owner August 7, 2026 22:32
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

This PR targets main — please retarget it to testing. All content PRs (features, fixes, BST changes) land on testing. The main branch is a release bookmark only.

castrojo
castrojo requested changes Aug 9, 2026
View reviewed changes

@castrojo castrojo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes — the intended test content looks reasonable, but this PR is not mergeable in its current state:

  1. Wrong base branch. The PR targets main; dakota policy (enforced by the on-pr-opened-or-updated workflow, which is currently failing on this PR) requires all content PRs to target testing. CI log: ERROR: PRs must target 'testing'. Got: 'main'.
  2. Merge state: CONFLICTING.
  3. Diff is 161 files / +9467/-3189. Because of the base mismatch, the diff is dominated by main↔testing branch divergence (workflows, elements, docs) rather than the described change. The actual payload — new tests/unit/confirm-verify_test.bats (393 lines) plus the test-unit Justfile target — is buried and unreviewable in this state.

Please rebase/retarget onto testing so the diff contains only the BATS tests and the Justfile target. The test plan itself (cancel-gate, unauthenticated fallback, fingerprint collection, 4 verify verdicts, journal consent + redaction) covers the right security surface for the #1136 hardening, and closing #1179 is worthwhile — happy to re-review a clean diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@castrojo castrojo castrojo requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@castrojo @bketelsen @ahmedadan @hanthor @axel-kaliff @daegalus @Xarianne