Community verification — ujust verify

[castrojo]

Summary

Users who experience a bug should be able to verify that a fix works on their hardware and share that evidence with the community. Today ujust report captures diagnostic state for bug filing, but there is no structured path for "I upgraded and the bug is gone."

Design principles

• User controls everything. No data leaves the machine without explicit consent and review.
• Donation, not collection. Users donate their diagnostic data to their own gist. We never phone home.
• Simple verbs. ujust report files a bug. ujust verify 531 confirms a fix.
• Targeted, not generic. A verification report for a Tailscale bug should check Tailscale, not dump the whole system.
• Real verification available. Users who want to go deep can build from source and boot a VM to confirm the fix with their own eyes.

Proposed flow

1. User files bug → ujust report → gist URL in issue
2. Fix merges → ships in nightly
3. Maintainer comments with verify-steps (what to check)
4. Community member reboots → ujust verify <issue-number>
   a. Fetches issue title + verify-steps from GitHub
   b. Shows steps to the user: "please confirm these work"
   c. Collects targeted diagnostics (image digest, affected service status)
   d. User reviews, confirms consent
   e. Uploads evidence to their own gist
   f. Posts compact summary comment on the issue
5. Maintainer sees N verifications → confidence to close

Build-and-verify path (full confidence)

For users who want to confirm a fix before it ships in nightly, or who want to verify on their exact hardware with full control:

# 1. Check out the PR branch with the fix
git clone https://github.com/projectbluefin/dakota && cd dakota
git fetch origin pull/NNN/head:pr-NNN && git checkout pr-NNN

# 2. Build the image from source
just build default

# 3. Boot it in a VM on your machine
just boot-test              # automated: exits 0/1
just boot-fast              # interactive: SSH into ephemeral VM
just boot-vm                # full: QEMU with display

# 4. Test the specific behavior
# (verify-steps from the issue tell you what to check)

# 5. Report back
ujust verify <issue-number>   # from inside the VM, or after rebooting into the image

This gives power users and contributors a complete loop: build → boot → test → attest. The Justfile makes every step a single command.

Result states

Verification should produce one of:

• ✅ fixed — user had the bug, upgraded, bug is gone
• ❌ still broken — user upgraded, bug persists
• ⚠️ different — new or changed behavior
• ℹ️ cannot confirm — user did not experience original bug, just reporting state

Out of scope

• No telemetry, metrics pipelines, or phone-home
• No automatic data upload without user review
• No OTel collection during verify (keep it light)

Acceptance criteria

• ujust verify <issue-number> exists and is interactive
• Fetches issue context (title, verify-steps if present)
• Collects: image digest, kernel, affected service status, relevant journal slice
• User reviews all output before anything leaves their machine
• Evidence goes to user-owned gist
• Compact summary posted as issue comment (with gist link)
• Works without gh auth (saves locally, shows manual post command)
• Handles missing verify-steps gracefully (falls back to generic system state)
• Verify-steps can include "build from source" path for contributors who want full confidence
• just boot-test provides automated pass/fail for the build-and-verify path

Dakota · issue pipeline

  ✓  triage     needs kind/ + area/ then /approve
  ·  discussing —
  ▶  queued     —
  ·  claimed    —
  ·  done       —

area: area/ujust   priority: priority/p0
maintainer: —
reporter: comment /claim to fix it yourself, or wait for a contributor