Skip to content

[Dream Cycle 2026-08-30] compiler-parity: STEP 17-18 hard-codes gh gist create with no fallback + config-schema,golden-snapshots scan #54

Description

@ruvnet

Rotation

DAYINT 20260830 % 5 = slot 0 → DEEP=compiler-parity, SCAN=config-schema,golden-snapshots.

Ledger Check

docs/dream-cycle/LEDGER.md has 6 rows (2026-08-13 → 2026-08-26). ledger signals reports zeroMergeStreak: true (0 of last 14 candidate PRs merged) → biased tonight's candidate to tiny/one-parameter/easily-reviewable, per STEP 1.1. ledger verify independently reports 9 schema errors on the existing rows (evaluated: "partial" not in yes|no|blocked; combined verdicts like "ACCEPT / INCONCLUSIVE") — already tracked by issue #48 / PR #49, not re-filed here.

gh CLI is not available in this session's shell (which gh → not found); GitHub state was read/written via MCP tools instead.

Deep Dive (compiler-parity)

packages/compile/src/index.tsstep17to18Publish() unconditionally instructs: Otherwise `gh gist create --public` the report, capture GIST_URL. No fallback path is described, and no other gist-creation tool is offered. This is a parity bug: the compiled prompt assumes tooling (gh CLI) that this repository's own real nightly-runner environment does not have.

This is not hypothetical — it already happened once: docs/dream-cycle/2026-08-13-security-adversarial-report.md (lines 229-234) records that night's session had no gh and no gist-creation MCP tool, and improvised GIST=LOCAL (commit the report into the PR instead). Tonight's session hit the exact same wall (confirmed via which gh and a live tool search for a gist-creation MCP tool — neither exists). The compiler was never corrected after the first occurrence, so the workaround had to be rediscovered from scratch instead of being encoded once in the one place (@dream-machine/compile) that this repo's own header comment calls the single source of truth ("Do not hand-edit — change the config and recompile").

Hypothesis

Given the Dream Machine's self-hosted compiler, when STEP 17-18 is changed from unconditionally requiring gh gist create --public to describing gist publication as best-effort across whatever tool the environment actually provides (with an explicit GIST=LOCAL fallback, matching the fallback already proven safe on 2026-08-13), then future gh-less nights should no longer need to reinvent the fallback, relative to the current baseline where it exists only as one-off prose in a single past report — subject to: no other step's semantics change, tests stay green, and ACCEPT/REJECT/INCONCLUSIVE / no-self-merge / no-fabricated-GIST_URL invariants are untouched.

Evaluation Receipt

Real evaluator: npx vitest run.

  • Baseline (parent 7933c359): Test Files 7 passed (7), Tests 98 passed (98).
  • Candidate: Test Files 7 passed (7), Tests 99 passed (99) — 98 baseline + 1 new pinning test ('treats missing gist tooling as best-effort, not a stop condition'), zero regressions.
  • Golden snapshot regenerated; diff is exactly the STEP 17-18 paragraph.
  • Self-hosting closure: re-compiled this repo's own dream.config.json post-fix and confirmed the corrected wording is what tonight's actual routine prompt now contains.

EVALUATED=yes.

Darwin Results

Not run — no numeric fitness landscape for a documentation/prompt-text fix; skip recorded explicitly rather than fabricated.

Evidence

  • OBSERVATION: which gh → absent; live MCP tool search for "gist create" → no result.
  • OBSERVATION: docs/dream-cycle/2026-08-13-security-adversarial-report.md independently hit the identical gap 17 nights earlier.
  • MEASUREMENT: npx vitest run 98/98 → 99/99, exact command output captured in the committed report.
  • DECISION: encode the fallback into the compiler itself rather than leaving it as recurring per-night improvisation.

Reward-Hack Check

Self-adversarial pass (no second agent spawned — recorded limitation): no assertions removed (one added); no gold answers exist for this surface; the new test explicitly asserts the old unconditional sentence is gone, so a revert would be caught; evaluator unmodified; no threshold/cache changes; change is ~10 lines. No unresolved signal.

Security Review

No security-sensitive surface: no prompt-injection vector, no MCP/tool authority change, no credential path touched, no filesystem/network scope change, snapshot regenerated deterministically from source (not hand-edited).

Scan Findings

config-schemavalidateConfig() still doesn't validate EvaluatorEntrypoints values are non-empty strings, labels/competitors/extraDisciplines are string arrays, or that DreamSlot.scan has exactly 2 entries (type says [string,string] | string[], only <1 warns). Adjacent gaps already tracked by open PRs #29 (adrConvention pad/dir) and #39/#40 (evaluatorEntrypoint positional) — not re-filed to avoid duplicating in-flight work.

golden-snapshots — the only snapshot covers the metaharness fixture; there is no golden snapshot of this repo's own dream.config.json compiled output, so a regression in the actually-executed self-hosted prompt would only be caught indirectly. Follow-up, not fixed tonight.

Competitors

System Missing-tool handling Grade
Sakana AI "The AI Scientist" Local-disk paper artifact by default, no hard external-publish dependency A
OpenHands (OpenDevin) Sandboxed action/observation contract; missing tool → explicit ErrorObservation A
SWE-agent Declared tool bundles; missing tool is a config-time error B
DSPy/GEPA No hard dependency on any single external publishing tool B
AutoGPT-lineage Failure catalog documents silent-assumption failures — tonight's exact pre-fix failure mode C

Gist

Published as a committed artifact instead of a gh/MCP gist (no such tool available tonight — the exact gap this candidate fixes): docs/dream-cycle/2026-08-30-compiler-parity-report.md. GIST=LOCAL.

Witness

report_sha256: 53947cfb707b7102627113c64687bb04e24adea7648ad2977447cbdd609c1e8d
session_commit: 7933c3599abe22df5290f4609d1f93f598feb3de
witness: c1fe7926ffa1cd4cb49688c30de3115622582682948c001c935fce9d5f664eb5

Verify: sha256sum docs/dream-cycle/2026-08-30-compiler-parity-report.md then printf '%s%s' "<that hash>" "7933c3599abe22df5290f4609d1f93f598feb3de" | sha256sum must equal the witness above.

Recommendation

Human review of the draft PR. evaluated: accepted.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions