Promote 6 process improvements from the Socraates Ascent project - #6
Open
syriley wants to merge 6 commits into
Open
Promote 6 process improvements from the Socraates Ascent project#6syriley wants to merge 6 commits into
syriley wants to merge 6 commits into
Conversation
…#3) Extends the skill's own prerequisite rule one stage later. That rule already refuses a verbal assurance for setup - a credential is verified by a direct provider call, "and not by the engineer saying it works" - but step RESULTS were still recorded from the reported verdict alone. Where a step's expected outcome leaves a readable trace (a database row, a log line, an audit event, a rendered element), read it and record the result from that evidence alongside the verdict. If the trace contradicts the report, record NEITHER: put the contradiction to the engineer with the readings it allows. Where there is no readable trace, record the report and say so, so a later reader knows which results rest on evidence. A reported verdict stays irreplaceable - only the tester can say whether a screen made sense or whether real third-party software accepted a real input. It is just not a reliable signal for whether the system did the thing. Worked example from the contributing project: the final step of an MFA intent - an un-enrolled user on a policy-bound company cannot reach the app - was reported Pass while the store showed a pending credential, zero recovery codes, and no enrolment log line. The same evidence allowed two readings, a fail-open defect in the sign-in gate or an unfinished step. It was the latter and passed on a re-run, but recording the reported Pass would have closed the intent with its most security-critical assertion untested. Source: Ascent, Company Owner & MFA intent retro, 2026-08-02. (cherry picked from commit 8a5a15d)
Promote a process improvement from the Ascent project (Bolt 16 retro). When the preview/automation browser is unreliable during a UI-heavy UAT, allow UAT to be evidenced by the intent's deterministic E2E in the authed app plus rendered-surface screenshots, method recorded in script + sign-off. Bounded to tooling failure; not a substitute for hands-on stakeholder validation when available. Source: Ascent retro 2026-07-21 Bolt 16 (Fund UI). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 2f05543)
) Promote a process improvement from the Ascent project (Bolt 20 retro). Two agent sessions on one working tree collide on unit/bolt numbers and on shared cross-cutting files (schema, migration journal, shared domain module), and interleaved uncommitted state can't always be split cleanly. Guidance: reserve number blocks + re-check before file creation; serialise shared data-layer edits; better, isolate streams on separate git worktrees/branches. Source: Ascent retros 2026-07-21 Bolts 16 + 20. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit fd7b200)
The skill already rejects vague thresholds ("faster", "more secure") and already
traces end-to-end thresholds. Neither catches a threshold that is numeric,
testable, traced — and still measuring someone else's bug.
Source case: a bolt removed a per-suite database-migration race in a test suite.
Its AC was "3 consecutive green full-gate runs, zero hook/test timeouts". The race
died completely — zero hook timeouts across every run, against 14/6/10+ per run at
baseline, with the lane also 2.3x faster. But a green whole-gate result is a
COMPOSITE that aggregates every unrelated defect, and two pre-existing flakes the
unit never touched kept the run red. The work succeeded and the threshold failed,
discovered at verification — the most expensive moment for a scoping error,
because the bolt must then go Blocked and the threshold be renegotiated.
Worth noting what did NOT save it: the two out-of-scope suites were named in the
unit's Scope section before execution, with an explicit warning that a green run
would not mean they were fixed. Declaring them out of scope did not stop the AC
from depending on them — scope and threshold must agree, and only the threshold
closes the bolt.
Also records that a threshold narrowed after measurement must keep its original
wording, the reason it failed, and who decided.
(cherry picked from commit 0688147)
…be a CHECK (#5) The Post-Retro Improvement Workflow reliably turns findings into prose: a rule in the code standards, a bullet in a checklist. Prose is necessary — it carries the reasoning a test cannot — but it is not a mitigation. It only fires if the next person reads it, recognises their situation in it, and acts. Nothing in the ceremony currently asks the question that converts a hazard into a control. Source case, and the reason this is worth a step of its own. A project's retro recorded, accurately and in the right file, that a per-suite database-migration call "races on a fresh DB". Over the following ten days the suite grew to 51 files carrying exactly that pattern, because a sentence stops nobody. The race then cost a real diagnosis: four consecutive gate runs red at 18/12/8/15 files, never the same set, zero assertion failures — and because a red gate no longer carried information, a genuine dependency regression had to be found by bisecting against main instead of by reading the gate. The contrast inside the fixing bolt is the evidence. It added a drift-guard test that fails the gate on any reintroduction. While that bolt was still being verified, a concurrent session merged a 52nd instance — and the guard named it in six seconds. The sentence had been true for ten days and changed nothing; the check was true immediately and stayed true. Step 1.5 therefore requires each improvement to be recorded as "check landed" (naming the test/lint/gate step, confirmed SEEN to fail against the defect it prevents), "check possible, not landed" (with the reason, raised as an action so it is a decision rather than an omission), or "prose only — not mechanically decidable", which is the honest answer for judgement-shaped lessons and where most process improvements will land. The failure this catches is the prose-only improvement whose invariant was mechanically decidable all along. (cherry picked from commit 371ea1f)
… the set grows (#6) Where a unit applies the same scripted edit across many files, its safety rests on a pre-check performed on THOSE files — that they all share the shape the script assumes. That premise expires silently, and the script reports success either way. Source case: a sweep removed a per-suite migration call from 51 test files. A pre-check had established that all 51 used the child_process import solely for that call, so the script removed the import unconditionally — correct for all 51. A concurrent session then merged a 52nd file while the work was being verified, and re-running the same sweep over the rebased tree deleted an import that file still needed: it used the import a second time, for an unrelated guard. The script was not wrong; its premise had expired. Reading the diff caught it; the type checker would have caught it one wasted gate run later. The expensive version of this failure is a sweep whose broken assumption produces behavioural rather than compile-time damage. NOTE ON ADAPTATION: the draft recorded in the source project appended this after that project's call-site-enumeration guidance in Step 2. This file does not carry that material, so the paragraph has been rewritten to stand alone and anchored to the end of Step 2 instead. No claim in it depends on the absent text. (cherry picked from commit ecfee64)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six process improvements from the Ascent project (Socraates), each drawn from a named retro with a worked example. All six are already merged into
Socraates-Technologies/ai-dlc-base(PRs #1–#6) and are offered here for the shared rule set.Branched off current
upstream/mainand cherry-picked, so this is +40/-1 across 5 files and nothing else — no fork-specific history, no conflicts (including insetup-guide.md, which upstream also touched).What each change adds
skills/uat.mdskills/uat.mdsetup-guide.mdskills/nfr-bolt.mdskills/knowledge-promotion.mdskills/bolt-risk-assessment.mdThe two worth reading properly
knowledge-promotion.mdStep 1.5 is the one we'd most want in the shared set, because it addresses a failure mode of the ceremony itself. A retro recorded — accurately, in the right file — that a per-suite database migration "races on a fresh DB". Over the next ten days the test suite grew to 51 files carrying exactly that pattern, because a sentence stops nobody. The race then cost a real diagnosis: four consecutive gate runs red at 18/12/8/15 files, never the same set, zero assertion failures, and a genuine dependency regression had to be found by bisecting againstmainbecause the gate's reds no longer carried information.The fixing bolt added a drift-guard test. While it was still being verified, a concurrent session merged a 52nd instance — and the guard named it in six seconds. The sentence had been true for ten days and changed nothing; the check was true immediately. Step 1.5 makes every improvement declare check landed / check possible, not landed / prose only — not mechanically decidable, so a mechanically-decidable invariant can't quietly ship as prose.
uat.mdcorroboration corrects an incentive rather than a rule: an agent that wrote the code has every reason to accept a reported Pass and move on. In the source case the final step of an MFA intent — an un-enrolled user on a policy-bound company cannot reach the app — was reported Pass while the database showed a pending credential, zero recovery codes and no enrolment log line. It turned out to be an unfinished step, not a fail-open defect, and passed on a re-run — but recording the reported Pass would have closed the intent with its most security-critical assertion untested.Note for reviewers
Each change is one commit with its full reasoning in the message, so they can be taken individually if you'd rather not take all six. The
bolt-risk-assessment.mdparagraph was adapted from its original draft: that draft appended after call-site-enumeration guidance which exists in the source project but not here, so it was rewritten to stand alone — worth reading on its own terms rather than against the source project's version.