-
Notifications
You must be signed in to change notification settings - Fork 51
feat(prompt-preset): agent-first Grok 4.5 worker profiles #629
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 7 commits
968de55
0b9d69f
be5978f
7937eff
4ec38cb
71b30d2
fde7263
561d39e
e87155a
c01435d
dc177ae
7d251c5
8754a66
dc0aa9e
63860c5
a2739f7
fd67d99
b085061
3d88a60
c816650
d8e9854
2552ca7
966eb1f
f69cb4b
f1221dd
f91f552
2997f16
9a69949
b11e8a6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -264,29 +264,32 @@ | |||||||||
|
|
||||||||||
| ## Grok 4.5 preset (unreleased — 2026-07-17) | ||||||||||
|
|
||||||||||
| Grok 4.5 has **not** been formally merged. Do not invent `v1`/`v2`/… edition labels for unreleased retunes — keep a single current section for this feature until it lands. | ||||||||||
|
|
||||||||||
| ### What changed (current branch state) | ||||||||||
| - `grok-4.5.ts` (2026-07-28, diet): CEO core compressed from 4606 to 3832 template characters (~17% cut) with zero behavior removal, grounded in xAI Grok 4.5 guidance (docs.x.ai/developers/grok-4-5; the grok-code prompt-engineering guide): Grok 4.5 follows terse, structured instructions without repeated emphasis and is trained for tool-loop reliability, so triplicated rules were merged into single homes. Specifically: the audit rules (Role bullet + Operating Loop step 4 + Verification section) collapsed into one **Audit** bullet; the human-surface/report contract (intro + Role bullet + Output) into intro + **Output**; Intent-gate/ask-one-question (Intent Gate + Loop step 1) into **Intent Gate**; plan/todo (Loop step 2) and parallel delegation (Loop step 3) into the **Delegate** bullet; Oracle review (Role bullet + Loop step 5) into the **Consult Oracle** bullet. The `## Operating Loop` and `## Verification` headings are gone; every unique rule they carried survives. All preset-test anchors unchanged and green. | ||||||||||
| - `grok-4.5.ts`: rewritten as a full-core preset via the `corePrompt` override (same shape as `gpt-5.5.ts` / `gpt-5.6.ts`). The role is now **CEO / orchestrator**, not a sibling tuningSection: Grok 4.5 acts as the single human-facing surface, delegates implementation work to background worker subprocesses spawned via `bash` as `senpi --print -p "..." --model <worker>` invocations (background `&` for parallel, output to temp files, `read` to collect), framed against GPT-5.6 prompting doctrine (implement-don't-propose, Manual QA Gate, binding stop contract). It consults a separate `senpi --print` review invocation before deploying non-trivial changes (the Oracle pattern), audits worker evidence rather than relaying self-report, and reports synthesized outcomes to the user. Trivial one-line fixes stay direct. | ||||||||||
| - senpi does NOT expose a `task` / `subagent` / `spawn` tool to the model - the built-in tool surface is bash/edit/read/write/grep/ls/find. So the CEO delegates through the concrete primitive it has (`bash` spawning `senpi --print` subprocesses), mirroring the gpt-5.6.ts rule of never naming tools that do not exist here. An earlier draft of this preset referenced a `task` tool with `category: "deep"` / `"ultrabrain"` values; that was a defect (those are the *orchestrator-side* task tool's categories, not anything the senpi agent exposes to Grok), and the regression test now explicitly pins that those names do not appear in the preset. | ||||||||||
| - Reuses `buildTestDisciplineSection()` and `buildFileOperationsTuning()` so shared rules stay single-sourced. Dynamic pieces (tool section, context files, skills, date, cwd) still come from `buildDynamicSystemPrompt`. | ||||||||||
| - Prior tuningSection content (act-once-context-sufficient, claim-auditing, no-promise-endings, context-limit continuation) was superseded by the CEO core, which subsumes those rules into the CEO's audit + reporting duties and the binding Stop Goal. The Mario benchmark rationale is preserved below for history. | ||||||||||
| - Benchmark evidence from the prior tuningSection version is under `local-ignore/qa-evidence/20260717-grok45-mario-benchmark/`. | ||||||||||
| - `presets.ts`: `hasGrok45Signal` / `isGrok45Model` unchanged (match any Grok 4.5 id shape without catching `grok-4.3` / `grok-4.20-*` / `grok-3`). | ||||||||||
| - `settings.ts`: `"grok-4.5"` joins `PromptPresetName` / `VALID_PRESETS` (unchanged). | ||||||||||
| - `test/suite/prompt-presets-grok-4-5.test.ts`: id resolution, negative neighbors, settings force, and catalog coverage unchanged. The old tuning-string regex pins and the 900–1800 character tuning-size guard were replaced with CEO-signal assertions (acting as the CEO and orchestrator; delegate implementation to background workers via `bash`; `senpi --print`; GPT-5.6 prompting doctrine; implement-don't-propose; Manual QA Gate; consult Oracle before deploying; you are the human surface; Stop Goal; STOPPING IS MANDATORY AND IMMEDIATE; `apply_patch` and `### Test Discipline` present; routing-line preserved). Also pins that the preset does NOT name a nonexistent `task`/`category`/`run_in_background` tool. | ||||||||||
| ### Agent-first Implementer/Oracle profiles (2026-08-01) | ||||||||||
|
|
||||||||||
| ### Why | ||||||||||
| - The CEO role is not a small addendum on top of the default identity — it is a different operating posture (orchestrator + human surface, not implementer), which the `tuningSection` shape cannot express. The `corePrompt` override is the documented path for full-role rewrites (per `AGENTS.md` and the gpt-5.5/5.6 precedent). The Mario benchmark established that evidence-grounded continuation and claim-auditing are the right Grok 4.5 execution discipline; the CEO core subsumes those into the CEO's audit + reporting duties and the Stop Goal rather than duplicating them. | ||||||||||
| - Delegation framing against GPT-5.6 doctrine is chosen because the gpt-5.6 preset already encodes that doctrine for the implementation-worker role; the CEO points its worker children at the same doctrine so worker behavior matches what gpt-5.6 would do in-session. | ||||||||||
| #### What changed | ||||||||||
| - `grok-4.5.ts` Role section: dropped the sole `--model gpt-5.6*` implementer path and the "gpt-5.6 prompting guide loads doctrine automatically" coupling. | ||||||||||
| - Workers are **invocation profiles** supplied through each child's explicit `--system-prompt`, not tools or user-message-only personas: **Implementer** (workspace-writing executor) and **Oracle** (read-only analysis/high-risk review). Critic/Planner/Explorer remain CEO responsibilities. | ||||||||||
| - Implementer/Oracle doctrine is model-independent at system priority: both prohibit nested workers; Implementer owns edits/tests/Manual QA, while Oracle has a read-only tool allowlist and no shell. | ||||||||||
| - Spawn remains `bash` + `senpi --print`, now with private `umask 077` / `mktemp -d` transport, cleanup traps, separate output/status files, `env -i` environment minimization, ephemeral `--no-session`, disabled extensions/skills/context/templates/nested-agents/fallback, and per-role `--tools` allowlists. | ||||||||||
| - CLI `--system-prompt` / `--append-system-prompt` values are forwarded into the resource loader. An explicit replacement wins over model presets; explicit appends remain after the selected preset. This closes the recursive-Grok child path discovered during review. | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: The 2026-08-01 changelog section describes host-side work that is not part of this prompt-only PR: it states the CLI Prompt for AI agentsThere was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: The new changes.md section presents CLI prompt-precedence forwarding (--system-prompt / --append-system-prompt into the resource loader, main.ts and agent-session.ts prompt metadata) as work shipped in this PR, and lists main.ts / agent-session.ts as 'expected merge conflict zones' for it. But this PR is described as prompt-only (deferring all infrastructure), the batch contains only grok-4.5.ts/test/changes.md, and that CLI forwarding is already documented as pre-existing in src/changes.md. As written, the tracker mis-attributes infrastructure to a prompt-only change and points an upstream sync at files this PR never touched. Consider rewording these bullets to state that the CLI forwarding already exists upstream and is merely relied on (not changed) by this preset, or drop the main.ts/agent-session.ts merge-conflict zone. Prompt for AI agents
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: This changes.md section describes resource-loader / Prompt for AI agents
Suggested change
|
||||||||||
| - Oracle wording is high-risk final review / hard debug — not "before deploying". One orchestration level; workers must not re-delegate. | ||||||||||
| - Brief fields: ROLE, GOAL, SCOPE, CONSTRAINTS, DONE WHEN, RETURN. | ||||||||||
| - Tests pin effective prompt precedence, CLI option forwarding, isolated spawn requirements, id resolution, settings force, catalog sweep, and no fake task-tool API. | ||||||||||
|
|
||||||||||
| ### Why extension system couldn't handle this differently | ||||||||||
| - Preset selection and family tuning are owned by this builtin; no core prompt code changed. | ||||||||||
| #### Why | ||||||||||
| - User direction: prefer specifying worker **roles** over locking every implementation child to GPT. Model presets must not be the only carrier of execution doctrine under senpi's no-task-tool harness. | ||||||||||
| - Oracle/Momus/Metis review of the multi-agent plan: five named agents overbuilt; 2 profiles max; doctrine cannot depend on gpt-5.6 preset. Follow-up code review proved user-message briefs alone could not override child model presets, requiring the minimal CLI prompt-precedence support above. | ||||||||||
|
|
||||||||||
| ### Expected merge conflict zones on next upstream sync | ||||||||||
| - LOW: `presets.ts` Grok matcher / `settings.ts` union if upstream adds its own Grok preset. | ||||||||||
| - LOW: `grok-4.5.ts` wording and Grok test phrase pins. | ||||||||||
| #### Why extension system couldn't handle this differently | ||||||||||
| - The preset owns role selection, but explicit CLI prompt values were not forwarded and a per-turn preset otherwise replaced the loader prompt. The host must preserve the documented explicit prompt precedence before the preset can safely create role-specific child sessions. | ||||||||||
|
|
||||||||||
| #### Expected merge conflict zones on next upstream sync | ||||||||||
| - MEDIUM: `main.ts` resource-loader option forwarding and `agent-session.ts` prompt metadata. | ||||||||||
| - LOW: prompt-preset precedence, Grok Role wording, and focused tests. | ||||||||||
|
|
||||||||||
| Grok 4.5 has **not** been formally merged. Do not invent `v1`/`v2`/… edition labels for unreleased retunes — keep a single current section for this feature until it lands. The **Agent-first Implementer/Oracle profiles (2026-08-01)** subsection above is the current design. Historical notes below are retained only for provenance and are **superseded** by that retune (including Oracle wording: high-risk final review / hard debug, **not** "before deploying"; worker doctrine is model-independent, not gpt-5.6-only). | ||||||||||
|
|
||||||||||
| Historical implementation details remain in Git history and the earlier evidence directories; this section documents only the current unreleased contract. | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: This changelog edit removes the 2026-07-28 diet/CEO-core rationale that grok-4.5.ts's header comment still points to ('full rationale in changes.md ("Grok 4.5 preset" section)'), leaving a dangling cross-reference. Either retain a one-line pointer noting the diet rationale moved out of the doc, or update the grok-4.5.ts comment to stop referencing this section for it. Prompt for AI agents |
||||||||||
|
|
||||||||||
| ## Overview | ||||||||||
| Per-model prompt preset extension. Selects a tuned system prompt based on the active model and exposes it through the dynamic prompt builder. | ||||||||||
|
|
||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3: The new Agent-first changelog section contradicts the retained '### What changed (current branch state)' block in the same changes.md (it still says workers are framed against GPT-5.6 prompting doctrine, Oracle runs 'before deploying', and parallel
&spawn is fine), and the retained note that the test pins 'GPT-5.6 prompting doctrine' no longer matches the test (which asserts not.toMatch(/gpt-5.6 prompting guide/i)). Since this section is labeled as the current branch state, the stale/contradictory text will mislead future readers; consider updating or retiring it now that the agent-first retune lands.Prompt for AI agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed / already superseded in be5978f and re-checked in 7937eff.
There is no remaining present-tense
### What changed (current branch state)block. Historical notes are labeled superseded by the 2026-08-01 agent-first retune, and the current subsection is the source of truth for Oracle wording and model-independent doctrine.