Skip to content

fix(agent): harden model execution continuity - #262

Merged
steipete merged 11 commits into
mainfrom
codex/fix-ollama-agent-loop
Jul 11, 2026
Merged

fix(agent): harden model execution continuity#262
steipete merged 11 commits into
mainfrom
codex/fix-ollama-agent-loop

Conversation

@steipete

Copy link
Copy Markdown
Collaborator

Summary

  • pin each agent session to a credential-free provider-qualified model identity, preserve that selection across config drift, and fail closed for ambiguous legacy sessions
  • harden session storage with safe paths and atomic persistence, and preserve resumable state across cancellation, step-budget exhaustion, provider errors, and terminal events
  • make --max-steps count model turns, validate 1...100, finish pending tool work correctly, and report cancellation, usage, tool, and terminal events exactly once
  • preserve native Ollama reasoning/tool history across authenticated multi-turn calls without replaying provider metadata across an API-key boundary
  • validate tool capability for local/custom models, preserve custom-provider configuration on save, and route taskless session resume through chat with failing lookup errors
  • document the Ollama execution, streaming, privacy, and session-continuity contracts

Proof

  • pnpm run build:cli
  • pnpm run test:safe — 721 tests in 82 suites
  • full PeekabooCore suite — 1,084 tests in 147 suites
  • focused production command-runner regressions for model parsing, provider config, taskless resume, invalid sessions, turn boundaries, event lifecycle, cancellation, step budgets, and session continuity
  • SwiftFormat, changed-file strict SwiftLint, and git diff --check
  • autoreview of each bounded commit; accepted findings fixed for authenticated reasoning replay, credential-free test isolation, taskless resume routing/error propagation, and Ollama streaming documentation

Live Ollama validation

Ran the debug CLI against native ollama/qwen3.5:9b with a task requiring exactly one shell call to pwd, --max-steps 4, local-only routing, and verbose streaming. The run used OllamaProvider, streamed reasoning, invoked the shell exactly once, completed a second model turn, emitted the terminal event, reported two model turns, and returned the exact working directory.

Tachikoma prerequisites are already landed in openclaw/Tachikoma#37 and openclaw/Tachikoma#38; their exact-head CI matrices are green.

@steipete
steipete marked this pull request as draft July 11, 2026 03:08
@steipete

Copy link
Copy Markdown
Collaborator Author

Independent final-head audit found blockers after the initial review slices passed. Holding this PR in draft while fixing:

  • require a terminal stream event before any provider's tool calls can execute;
  • preserve terminal done / need_info content after narrated tool turns;
  • balance cancellation and skipped-tool lifecycle events, with explicit failure payloads;
  • keep resume dry-runs credential-free and redact persisted legacy model descriptions from errors;
  • preserve persisted session timestamps and strip all URL userinfo from endpoint identities;
  • propagate resumed-turn failures as nonzero CLI exits;
  • align offline models-provider behavior and taskless-resume docs with the implementation;
  • isolate new agent tests from the real ~/.peekaboo/sessions store.

I will update the branch with focused regressions, rerun autoreview, and restore ready status only after the exact head is green.

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codex review: found issues before merge. Reviewed July 10, 2026, 11:17 PM ET / July 11, 2026, 03:17 UTC.

Summary
The PR hardens agent model and session continuity, persistence, step budgets, event lifecycle, Ollama replay, provider configuration, CLI resume behavior, documentation, tests, changelogs, and the Tachikoma submodule pointer.

Reproducibility: yes. The exact source paths provide high-confidence reproductions for unterminated tool streams, narrated terminal tools, URL userinfo retention, and swallowed resumed-turn failures, and the collaborator author independently confirmed the combined-head blockers.

Review metrics: 3 noteworthy metrics.

  • Patch surface: 53 files, +4,859/-1,140. The branch crosses execution, persisted state, CLI, provider configuration, docs, tests, changelogs, and a dependency boundary.
  • Regression coverage: 7 new test files. Coverage is substantial, but confirmed combined-head integration paths remain broken or uncovered.
  • Dependency movement: 1 submodule pointer changed. The resulting behavior depends on paired Tachikoma commits and needs exact-head compatibility proof.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦞 diamond lobster
Patch quality: 🦪 silver shellfish
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Fix and regression-test every prior and author-confirmed exact-head blocker.
  • Rerun autoreview, CLI build, safe and full PeekabooCore suites, lint/format, and redacted live Ollama resume, terminal-tool, cancellation, and step-exhaustion proof.

Risk before merge

  • [P1] Merging this head can dispatch provider-requested tools from an incomplete stream, discard explicit done or need_info results, retain URL usernames in persisted endpoint identity input, and return success after a failed resumed chat turn.
  • [P1] The patch changes legacy-session routing, persisted identities and timestamps, checkpointing, step-limit semantics, provider configuration saves, and a Tachikoma gitlink, so fresh-run and upgrade/resume behavior require exact-head cross-repository proof.

Maintainer options:

  1. Complete the final-head repair (recommended)
    Keep the PR draft while the author fixes every acknowledged blocker and reruns focused suites, autoreview, and redacted live validation against the paired Tachikoma head.
  2. Split the risky surfaces
    If the combined patch remains difficult to prove, separate persisted model identity and session migration from event-loop, CLI, and provider-save changes.

Next step before merge

  • [P2] The collaborator author has claimed and is actively repairing the exact draft branch, so a separate ClawSweeper repair job would duplicate and conflict with current work.

Security
Needs attention: The unchanged head incompletely strips URL userinfo from persisted endpoint identities and still needs credential-free legacy-error and provider-replay boundary verification.

Review findings

  • [P1] Require a terminal stream event before dispatching tools — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+StreamProcessing.swift:162-164
  • [P2] Preserve successful terminal-tool content after narration — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+Streaming.swift:308-311
  • [P2] Strip the URL username from endpoint identities — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+ModelIdentity.swift:170-173
Review details

Best possible solution:

Finish the claimed exact-head repair with terminal-event gating, explicit terminal-result preservation, balanced lifecycle events, fully credential-free identities and errors, timestamp-safe persistence, nonzero resume failures, isolated tests, aligned docs, and paired Tachikoma validation.

Do we have a high-confidence way to reproduce the issue?

Yes. The exact source paths provide high-confidence reproductions for unterminated tool streams, narrated terminal tools, URL userinfo retention, and swallowed resumed-turn failures, and the collaborator author independently confirmed the combined-head blockers.

Is this the best way to solve the issue?

No for the current head. The continuity architecture and test investment are directionally strong, but the implementation is not the safest solution until the confirmed ordering, content-preservation, redaction, exit-status, lifecycle, timestamp, documentation, and isolation defects are repaired.

Full review comments:

  • [P1] Require a terminal stream event before dispatching tools — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+StreamProcessing.swift:162-164
    Reject every provider stream that ends without a terminal event before executing any collected tool calls. The current guard runs only when assistant text buffering is enabled, so a tool-call-only unterminated stream can still trigger side effects.
    Confidence: 0.99
  • [P2] Preserve successful terminal-tool content after narration — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+Streaming.swift:308-311
    Keep the explicit done or need_info result even when the model emitted narration first. The current empty-content check discards the terminal tool's authoritative message or question whenever earlier text is present.
    Confidence: 0.98
  • [P2] Strip the URL username from endpoint identities — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+ModelIdentity.swift:170-173
    Clear both username and password before canonicalizing and hashing provider endpoints. This code removes only the password, so credential or tenant-like username material remains part of persisted session identity.
    Confidence: 0.99
  • [P1] Propagate failed resumed turns to the process exit — Apps/CLI/Sources/PeekabooCLI/Commands/AI/AgentCommand+Chat.swift:351-365
    Make interactive resumed-turn failures reach the command exit status instead of being rendered and then swallowed by the chat loop. Automation invoking a resumed session must receive a nonzero exit when continueSession fails.
    Confidence: 0.96

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9be5918dac92.

Label changes

Label changes:

  • add P1: Confirmed exact-head defects can execute unintended tool work, lose terminal results, expose endpoint identity material, and mask failed resumed runs.
  • add merge-risk: 🚨 compatibility: The PR changes legacy-session resume rules, CLI exit behavior, max-step semantics, provider-save behavior, and a provider dependency.
  • add merge-risk: 🚨 session-state: The PR changes persisted model identities, timestamps, checkpoints, cancellation state, and resumability across failures.
  • add merge-risk: 🚨 security-boundary: Provider identity and reasoning replay cross credential boundaries, and URL userinfo redaction is incomplete on the current head.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains credible after-fix live Ollama CLI proof for provider routing, streamed reasoning, one real shell call, a second model turn, terminal completion, and expected output; this proof does not clear the separately confirmed combined-head defects.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body contains credible after-fix live Ollama CLI proof for provider routing, streamed reasoning, one real shell call, a second model turn, terminal completion, and expected output; this proof does not clear the separately confirmed combined-head defects.

Label justifications:

  • P1: Confirmed exact-head defects can execute unintended tool work, lose terminal results, expose endpoint identity material, and mask failed resumed runs.
  • merge-risk: 🚨 compatibility: The PR changes legacy-session resume rules, CLI exit behavior, max-step semantics, provider-save behavior, and a provider dependency.
  • merge-risk: 🚨 session-state: The PR changes persisted model identities, timestamps, checkpoints, cancellation state, and resumability across failures.
  • merge-risk: 🚨 security-boundary: Provider identity and reasoning replay cross credential boundaries, and URL userinfo redaction is incomplete on the current head.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body contains credible after-fix live Ollama CLI proof for provider routing, streamed reasoning, one real shell call, a second model turn, terminal completion, and expected output; this proof does not clear the separately confirmed combined-head defects.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains credible after-fix live Ollama CLI proof for provider routing, streamed reasoning, one real shell call, a second model turn, terminal completion, and expected output; this proof does not clear the separately confirmed combined-head defects.
Evidence reviewed

Security concerns:

  • [medium] Remove all URL userinfo from endpoint identities — Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+ModelIdentity.swift:170
    Canonical endpoint derivation clears the password but retains the username before hashing, allowing credential-like or tenant-identifying URL material to influence persisted session metadata and related diagnostics.
    Confidence: 0.99

What I checked:

Likely related people:

  • steipete: Peter Steinberger is the dominant contributor to the merged agent-runtime history, authored this continuity series, independently confirmed the exact-head blockers, and is actively repairing the draft branch. (role: feature owner and active branch owner; confidence: high; commits: ad012853de2d, 1de520549271, fb0b0fdb1eff; files: Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+Streaming.swift, Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+ModelIdentity.swift, Apps/CLI/Sources/PeekabooCLI/Commands/AI/AgentCommand+Chat.swift)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-11T03:11:55.287Z sha fb0b0fd :: found issues before merge. :: [P1] Require a terminal stream event before dispatching tools | [P2] Preserve successful terminal-tool content after narration | [P2] Strip the URL username from endpoint identities | [P1] Propagate failed resumed turns to the process exit

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb0b0fdb1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +222 to 223
provider: configuration.provider,
messages: state.messages,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sanitize streaming history before provider calls

This streaming path sends the persisted transcript verbatim. When a session containing provider-specific reasoning/thinking messages is resumed with an explicit different streaming model or provider, those .thinking entries are replayed instead of being filtered by sanitizedForProviderContext(...) as in runGenerationLoop, so providers can receive incompatible signed/native reasoning and reject or mis-handle the continuation. Pass the sanitized message list here using configuration.provider before calling streamText.

Useful? React with 👍 / 👎.


components.scheme = scheme
components.host = host
components.password = nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Strip userinfo from endpoint identity hashes

canonicalEndpointIdentity removes the password but leaves components.user in the URL that is hashed. For custom endpoints configured with basic-auth/userinfo or a token in the username part, the session identity still depends on credential material, so rotating that username breaks otherwise-valid session resume and persists a credential-derived fingerprint. Clear the user field before hashing the canonical endpoint.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 session-state 🚨 Merging this PR could lose, corrupt, stale, or mis-associate session or agent state. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Jul 11, 2026
@steipete
steipete marked this pull request as ready for review July 11, 2026 03:31
@steipete
steipete merged commit 41da02e into main Jul 11, 2026
5 checks passed
@steipete
steipete deleted the codex/fix-ollama-agent-loop branch July 11, 2026 03:31

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb0b0fdb1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

components.scheme = scheme
components.host = host
components.password = nil
components.fragment = nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Strip query strings from endpoint hashes

When a custom endpoint is configured with credentials in the URL query (for example ?token=... or ?api-key=...), components.string below still includes that query in the SHA-256 input. Since this value is persisted as modelEndpointIdentity and later compared on resume, rotating the token makes an otherwise unchanged session fail verification and leaves a credential-derived fingerprint in session storage. Clear components.query before hashing the canonical endpoint.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 session-state 🚨 Merging this PR could lose, corrupt, stale, or mis-associate session or agent state. P1 Urgent regression or broken agent/channel workflow affecting real users now. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant