Skip to content

Promote 6 process improvements from the Socraates Ascent project - #6

Open
syriley wants to merge 6 commits into
99x-Projects:mainfrom
Socraates-Technologies:promote/ascent-bolt-106-plus-uat
Open

Promote 6 process improvements from the Socraates Ascent project#6
syriley wants to merge 6 commits into
99x-Projects:mainfrom
Socraates-Technologies:promote/ascent-bolt-106-plus-uat

Conversation

@syriley

@syriley syriley commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Six process improvements from the Ascent project (Socraates), each drawn from a named retro with a worked example. All six are already merged into Socraates-Technologies/ai-dlc-base (PRs #1#6) and are offered here for the shared rule set.

Branched off current upstream/main and cherry-picked, so this is +40/-1 across 5 files and nothing else — no fork-specific history, no conflicts (including in setup-guide.md, which upstream also touched).

What each change adds

File Change Retro
skills/uat.md A step's result is corroborated, not taken from the report — where the expected outcome leaves a readable trace (a row, a log line, an audit event), read it and record the result from that evidence alongside the verdict. If they contradict, record neither. Company-Owner/MFA intent
skills/uat.md Tooling-failure fallback — how to evidence UAT for a UI-heavy intent when browser automation is unreliable, with the method recorded in the script. Bolt 16
setup-guide.md Concurrent sessions — reserve numbers + serialise shared files — two agent sessions on one working tree collide on unit/bolt numbers and on shared cross-cutting files. Bolt 20
skills/nfr-bolt.md Scope the threshold to what the change CONTROLS — a threshold can be numeric, testable, traced, and still measuring someone else's bug. Bolt 106
skills/knowledge-promotion.md Step 1.5 — ask whether the improvement can be a CHECK rather than a sentence. Bolt 106
skills/bolt-risk-assessment.md A mechanical sweep's premise expires when the set grows — re-run the sweep's pre-checks, not just the sweep. Bolt 106

The two worth reading properly

knowledge-promotion.md Step 1.5 is the one we'd most want in the shared set, because it addresses a failure mode of the ceremony itself. A retro recorded — accurately, in the right file — that a per-suite database migration "races on a fresh DB". Over the next ten days the test suite grew to 51 files carrying exactly that pattern, because a sentence stops nobody. The race then cost a real diagnosis: four consecutive gate runs red at 18/12/8/15 files, never the same set, zero assertion failures, and a genuine dependency regression had to be found by bisecting against main because the gate's reds no longer carried information.

The fixing bolt added a drift-guard test. While it was still being verified, a concurrent session merged a 52nd instance — and the guard named it in six seconds. The sentence had been true for ten days and changed nothing; the check was true immediately. Step 1.5 makes every improvement declare check landed / check possible, not landed / prose only — not mechanically decidable, so a mechanically-decidable invariant can't quietly ship as prose.

uat.md corroboration corrects an incentive rather than a rule: an agent that wrote the code has every reason to accept a reported Pass and move on. In the source case the final step of an MFA intent — an un-enrolled user on a policy-bound company cannot reach the app — was reported Pass while the database showed a pending credential, zero recovery codes and no enrolment log line. It turned out to be an unfinished step, not a fail-open defect, and passed on a re-run — but recording the reported Pass would have closed the intent with its most security-critical assertion untested.

Note for reviewers

Each change is one commit with its full reasoning in the message, so they can be taken individually if you'd rather not take all six. The bolt-risk-assessment.md paragraph was adapted from its original draft: that draft appended after call-site-enumeration guidance which exists in the source project but not here, so it was rewritten to stand alone — worth reading on its own terms rather than against the source project's version.

syriley and others added 6 commits August 3, 2026 14:47
…#3)

Extends the skill's own prerequisite rule one stage later. That rule already
refuses a verbal assurance for setup - a credential is verified by a direct
provider call, "and not by the engineer saying it works" - but step RESULTS
were still recorded from the reported verdict alone.

Where a step's expected outcome leaves a readable trace (a database row, a log
line, an audit event, a rendered element), read it and record the result from
that evidence alongside the verdict. If the trace contradicts the report,
record NEITHER: put the contradiction to the engineer with the readings it
allows. Where there is no readable trace, record the report and say so, so a
later reader knows which results rest on evidence.

A reported verdict stays irreplaceable - only the tester can say whether a
screen made sense or whether real third-party software accepted a real input.
It is just not a reliable signal for whether the system did the thing.

Worked example from the contributing project: the final step of an MFA intent
- an un-enrolled user on a policy-bound company cannot reach the app - was
reported Pass while the store showed a pending credential, zero recovery
codes, and no enrolment log line. The same evidence allowed two readings, a
fail-open defect in the sign-in gate or an unfinished step. It was the latter
and passed on a re-run, but recording the reported Pass would have closed the
intent with its most security-critical assertion untested.

Source: Ascent, Company Owner & MFA intent retro, 2026-08-02.
(cherry picked from commit 8a5a15d)
Promote a process improvement from the Ascent project (Bolt 16 retro).
When the preview/automation browser is unreliable during a UI-heavy UAT,
allow UAT to be evidenced by the intent's deterministic E2E in the authed
app plus rendered-surface screenshots, method recorded in script + sign-off.
Bounded to tooling failure; not a substitute for hands-on stakeholder
validation when available.

Source: Ascent retro 2026-07-21 Bolt 16 (Fund UI).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 2f05543)
)

Promote a process improvement from the Ascent project (Bolt 20 retro).
Two agent sessions on one working tree collide on unit/bolt numbers and on
shared cross-cutting files (schema, migration journal, shared domain module),
and interleaved uncommitted state can't always be split cleanly. Guidance:
reserve number blocks + re-check before file creation; serialise shared
data-layer edits; better, isolate streams on separate git worktrees/branches.

Source: Ascent retros 2026-07-21 Bolts 16 + 20.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit fd7b200)
The skill already rejects vague thresholds ("faster", "more secure") and already
traces end-to-end thresholds. Neither catches a threshold that is numeric,
testable, traced — and still measuring someone else's bug.

Source case: a bolt removed a per-suite database-migration race in a test suite.
Its AC was "3 consecutive green full-gate runs, zero hook/test timeouts". The race
died completely — zero hook timeouts across every run, against 14/6/10+ per run at
baseline, with the lane also 2.3x faster. But a green whole-gate result is a
COMPOSITE that aggregates every unrelated defect, and two pre-existing flakes the
unit never touched kept the run red. The work succeeded and the threshold failed,
discovered at verification — the most expensive moment for a scoping error,
because the bolt must then go Blocked and the threshold be renegotiated.

Worth noting what did NOT save it: the two out-of-scope suites were named in the
unit's Scope section before execution, with an explicit warning that a green run
would not mean they were fixed. Declaring them out of scope did not stop the AC
from depending on them — scope and threshold must agree, and only the threshold
closes the bolt.

Also records that a threshold narrowed after measurement must keep its original
wording, the reason it failed, and who decided.

(cherry picked from commit 0688147)
…be a CHECK (#5)

The Post-Retro Improvement Workflow reliably turns findings into prose: a rule in
the code standards, a bullet in a checklist. Prose is necessary — it carries the
reasoning a test cannot — but it is not a mitigation. It only fires if the next
person reads it, recognises their situation in it, and acts. Nothing in the
ceremony currently asks the question that converts a hazard into a control.

Source case, and the reason this is worth a step of its own. A project's retro
recorded, accurately and in the right file, that a per-suite database-migration
call "races on a fresh DB". Over the following ten days the suite grew to 51 files
carrying exactly that pattern, because a sentence stops nobody. The race then cost
a real diagnosis: four consecutive gate runs red at 18/12/8/15 files, never the
same set, zero assertion failures — and because a red gate no longer carried
information, a genuine dependency regression had to be found by bisecting against
main instead of by reading the gate.

The contrast inside the fixing bolt is the evidence. It added a drift-guard test
that fails the gate on any reintroduction. While that bolt was still being
verified, a concurrent session merged a 52nd instance — and the guard named it in
six seconds. The sentence had been true for ten days and changed nothing; the
check was true immediately and stayed true.

Step 1.5 therefore requires each improvement to be recorded as "check landed"
(naming the test/lint/gate step, confirmed SEEN to fail against the defect it
prevents), "check possible, not landed" (with the reason, raised as an action so it
is a decision rather than an omission), or "prose only — not mechanically
decidable", which is the honest answer for judgement-shaped lessons and where most
process improvements will land. The failure this catches is the prose-only
improvement whose invariant was mechanically decidable all along.

(cherry picked from commit 371ea1f)
… the set grows (#6)

Where a unit applies the same scripted edit across many files, its safety rests on
a pre-check performed on THOSE files — that they all share the shape the script
assumes. That premise expires silently, and the script reports success either way.

Source case: a sweep removed a per-suite migration call from 51 test files. A
pre-check had established that all 51 used the child_process import solely for
that call, so the script removed the import unconditionally — correct for all 51. A
concurrent session then merged a 52nd file while the work was being verified, and
re-running the same sweep over the rebased tree deleted an import that file still
needed: it used the import a second time, for an unrelated guard. The script was
not wrong; its premise had expired. Reading the diff caught it; the type checker
would have caught it one wasted gate run later. The expensive version of this
failure is a sweep whose broken assumption produces behavioural rather than
compile-time damage.

NOTE ON ADAPTATION: the draft recorded in the source project appended this after
that project's call-site-enumeration guidance in Step 2. This file does not carry
that material, so the paragraph has been rewritten to stand alone and anchored to
the end of Step 2 instead. No claim in it depends on the absent text.

(cherry picked from commit ecfee64)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant