[Agent Reliability] Scope binding before evidence promotion

[fscfede-beep]

Scope binding before evidence promotion

Independent agent-reliability note for discussion and review.

Failure mode

An agent can observe a real entity and still be wrong if it promotes that entity before proving it belongs to the exact scope the user asked about.

Evidence therefore needs both:

• entity provenance — did I really observe this object?
• scope provenance — did I prove this object belongs to the current task?

Authentic evidence without scope provenance can contaminate canonical state.

Fail-closed invariant

DISCOVERED
-> QUARANTINED
-> SCOPE_BOUND
-> IDENTIFIED
-> PRIMARY_EVIDENCE
-> DIRECT_BINDING
-> VERIFIED

There is no direct DISCOVERED -> TARGET transition. If scope is unresolved: SCOPE_UNRESOLVED / NO_RESULT.

Current public implementation

Canonical repository: RUMBO-IA/Rumbo.

The generic guard and Codex-oriented refinements are on public main through merged PRs including:

• research: executable scope-binding guard for agent evidence (clean-root candidate) #17 — base generic scope-provenance guard;
• research: Codex thread scope evidence (R64 clean-history candidate) #28 — Thread.environments selection adapter + sanitized Codex diagnostic probe;
• research: separate workspace roots from write authority #33 — correction that enforces runtimeWorkspaceRoots != writableRoots.

Current merged code:
https://github.com/RUMBO-IA/Rumbo/tree/main/research/agent-reliability/scope-binding

Historical PRs #15, #24 and #27 are closed without merge and are not current implementation authority.

Current research candidate

Draft PR #34 adds permission-authority provenance:
#34

It requires both:

authority.permission_source
== permissionEvidence.source

and:

authority.permission_profile_id
== permissionEvidence.activePermissionProfile.id

before effective writable roots can satisfy required write scope.

Fresh exact-head receipt for #34:

• base eee8cd79148430bbac57148152e691ec8f87aa2a;
• head 3db8cf123a428a5e93345c2db9ddac02e619cfe0;
• Scope Binding Reference run 33924090723: SUCCESS;
• regression suite: 44/44 PASS;
• Public privacy gate 33924090724: SUCCESS;
• Vercel: SUCCESS;
• research: bind write permission evidence to trusted source #34 remains DRAFT / NOT MERGED.

OpenAI / Codex relevance

Related open bug:
openai/codex#24224

Our contributions there are acceptance/diagnostic models only; they do not claim an identical root cause or a Codex fix.

Current model:

trusted project/workspace authority
+ selected environment evidence
+ fresh runtime/connection evidence
+ trusted permission-source evidence
+ server-resolved active permission profile
+ effective writable roots
-> executable/write authority

Key separations:

• selection is not connection;
• runtimeWorkspaceRoots are not write permissions;
• a non-empty source label is not trusted authority;
• observed state must not define or expand its own authority.

The current Codex app-server protocol exposes useful seams for this model: Thread.environments reports selected environments independently of connection status, while environment connection/status and permission-profile surfaces provide separate runtime/authority evidence.

Retraction and freshness

If a candidate is proven out of scope: redact the receipt that introduced it, reset the target to UNRESOLVED, close downstream gates, preserve negative memory, and require new independent scope evidence before reuse.

A control-plane online or selected-environment signal also does not prove executable availability; fresh runtime evidence is required.

Boundary

This is an independent reliability reference. It is not OpenAI endorsement, an upstream merge, a proven Codex root cause, or proof that a separate private product-specific guard is deployed.

Evidence authenticity is necessary but not sufficient. Scope provenance and authority provenance must be explicit before promotion.

[fscfede-beep]

Implementation update: the pattern is now reproducible in public PR #15. It adds a generic fail-closed scope-provenance validator plus 7 regression tests. Local verification: 7/7 PASS, git diff --check PASS, and a sensitive-data scan for WhatsApp/group/phone identifiers returned 0 matches. The README also links adjacent OpenAI Agents SDK design guidance on scoped state isolation and on not inheriting evidence merely from conceptual similarity; those references are context, not a claim of the same OpenAI bug. The private control-plane version is separately protected by candidate-validation and passed 64/64 tests. Public PR: #15

[fscfede-beep]

Verification update: public PR #15 is now CI-backed. Exact head a7efa66. The dedicated scope-binding test job passed, the existing privacy gate passed, and the workflow was corrected to avoid duplicate push+PR runs. The public example remains sanitized and generic; no WhatsApp/group/phone identifiers are present. Adjacent OpenAI design references that motivated the comparison (not evidence of the same bug): https://github.com/openai/openai-agents-python/blob/89c02c828ee8510fe9a84ee6675608193aa13b02/.agents/references/agent-definition-and-run-context.md and https://github.com/openai/openai-agents-python/blob/89c02c828ee8510fe9a84ee6675608193aa13b02/.agents/skills/maintainer-review/references/evaluation-framework.md . I am deliberately not opening an upstream OpenAI bug from this evidence alone because there is not yet a reproducible defect in an OpenAI repository; this thread is a reference pattern and implementation.

[fscfede-beep]

Upstream relevance update: I found a materially related open Codex report, openai/codex#24224 (concurrent sessions leaking workspace roots across projects). I added a single comment there with a fail-closed workspace-scope admission invariant and regression matrix, explicitly without claiming the same root cause or a Codex fix: openai/codex#24224 (comment) . Public reference PR #15 now links #24224 in its README; exact head b9a05c2 passed the dedicated test job and existing privacy gate. This keeps the public thread tied to a real upstream reliability case rather than a hypothetical OpenAI bug claim.

[fscfede-beep]

R43 implementation update: PR #15 now includes a workspace-specific admission guard for the open Codex isolation case #24224. Exact head cc58825. Local suite: 16/16 PASS. GitHub Actions scope-binding test: PASS. Privacy gate: PASS. The adapter rejects stale session CWD, foreign workspace roots, foreign writable roots, transport-state races, and contaminated sessions; explicit task-local auxiliary roots are allowlisted. I updated the existing upstream comment in openai/codex#24224 in place rather than adding another comment: openai/codex#24224 (comment) . It still explicitly does not claim the same root cause or a Codex fix.

[fscfede-beep]

R3 public-reference update: PR #15 now includes a workspace-scope admission adapter tied to the concrete openai/codex#24224 failure class. The stronger model separates trusted scope authority from observed session state, so a session cannot prove itself correct merely because cwd/workspace/writable-root fields agree with each other. Exact head: 356b53e. Fresh CI Scope Binding Reference: PASS; Public privacy gate: PASS; 22 tests ran, OK. Source-seam analysis was added for current openai/codex snapshot b27a6321: session_flow.rs applies session cwd/runtime workspace roots to config and permissions before skills refresh/initial-message submission; turn_context.rs exposes active cwd/workspace roots; thread_settings.rs persists thread-owned workspace settings around compaction checkpoints. This narrows the proposed admission/repair boundaries without claiming an upstream root cause or fix. No private WhatsApp/group/phone identifiers are present.

[fscfede-beep]

Self-audit correction: the first workspace adapter still trusted too much of the observed session. A fully contaminated but internally consistent state could pass, and the observed receipt could attempt to self-allowlist an unrelated auxiliary root. PR #15 now separates a trusted scope_authority (from a distinct project-selection boundary) from observed session state; observed metadata cannot define or expand that authority. The test file itself also had a discovery defect: duplicate tests existed after the module's main block, so they were present in text but not executed. That was corrected and discovery is now counted explicitly. Current exact head: 356b53e; dedicated CI run 33707976262 PASS; 22/22 discoverable regression tests PASS; privacy gate PASS. I also independently verified the Codex source-seam note against pinned upstream source and edited the existing openai/codex#24224 comment in place rather than adding another message. This correction is part of the point of the exercise: a reliability control should fail its own audit before it is presented as durable.

[fscfede-beep]

Status reconciliation — 2026-09-03

PR #15 is now closed without merge. It remains a historical, sanitized reference for the scope-binding pattern discussed here; it is not canonical RUMBO product code, not a production deployment, and not a currently promoted implementation.

The issue remains open as an agent-reliability design discussion. Any future implementation claim should be tied to a new exact commit/run rather than the historical PR state above.

[fscfede-beep]

R46 / Thread.environments adapter candidate (2026-09-04): OpenAI Codex commit 2b554fd3f96a128be52e0d64b01f6adf16cc467a added experimental Thread.environments with environmentId, cwd and runtimeWorkspaceRoots, explicitly documenting that this reports selection rather than connection status. I added a narrow TDD-built adapter in PR #24 that treats this field as selection evidence only. RED was observed before implementation (missing adapter import); GREEN exact head dd23536 passed Scope Binding Reference run 33856272997 with 32/32 tests. New regressions prove: matching selection without runtime proof => SELECTION_ONLY; wrong environment => fail closed; matching independent authority + selection + runtime proof => RUNTIME_SCOPE_BOUND. Sensitive scan: 0 matches. PR #24 remains OPEN / NOT MERGED. No claim that this fixes openai/codex#24224 and no additional upstream comment was posted because there is no new maintainer/user response to our existing contribution.

[fscfede-beep]

2026-09-04 continuation / authority-provenance refinement: new evidence in openai/codex#24224 reports that cross-project contamination can reach persisted effective writable roots, not only cwd/workspace_roots (issue comment 5152365265). Public main already contains the first corrective boundary via merged PR #33: runtimeWorkspaceRoots are scope/runtime evidence only and do not imply writableRoots. Exact merged main 35fa6e5 passed Scope Binding Reference CI run 33908927547 with 40/40 tests and Public privacy gate run 33908927603 SUCCESS. Self-audit then found a second provenance gap: permissionEvidence.source was accepted if merely non-empty, so observed state could still self-certify write authority with an arbitrary source label. TDD reproduced both false positives: (1) mismatched self-declared permission source, and (2) permission evidence with no externally authorized source identity. Draft PR #34 fixes this by requiring authority.permission_source and exact equality with permissionEvidence.source before writableRoots can satisfy required write scope. Exact head 7d6aa6d; Scope Binding Reference run 33922801293 SUCCESS with 42/42 tests; Public privacy gate 33922801141 SUCCESS. PR #34 remains DRAFT / NOT MERGED. The existing upstream comment in openai/codex#24224 was edited in place rather than adding another comment. It now records the stronger model: trusted scope authority + selected environment evidence + fresh runtime evidence + independently authorized permission evidence -> executable/write authority. No claim of identical root cause, OpenAI acceptance, or Codex fix.

Structured permission-profile provenance refinement

Further source inspection of current Codex app-server confirms that ThreadSettings exposes both server-reported activePermissionProfile provenance and sandboxPolicy; the workspaceWrite sandbox policy contains effective writableRoots. That gives the reference a concrete server-owned evidence seam instead of a free-form source label.

PR #34 was refined in place. It now requires both:

authority.permission_source == permissionEvidence.source
authority.permission_profile_id == permissionEvidence.activePermissionProfile.id

before writable roots can satisfy required write scope.

TDD added two more RED cases: mismatched active profile id and missing active-profile provenance. Exact current head 52d64a8a9a84ee3543d54bca5b52baadb5ceba1e; Scope Binding Reference run 33923301159 SUCCESS with 44/44 tests; Public privacy gate 33923301311 SUCCESS; sensitive/retracted identifier scan 0. PR #34 remains DRAFT / NOT MERGED.

[fscfede-beep]

R75 current-state reconciliation (2026-09-05 UTC):

• canonical public repository is now RUMBO-IA/Rumbo (repository id unchanged: 1353125296); old personal-namespace URLs redirect but are no longer canonical references;
• public main is eee8cd79148430bbac57148152e691ec8f87aa2a, protected with required privacy + Vercel checks;
• current-main Public privacy gate run 33938382504: SUCCESS; Vercel status: SUCCESS;
• merged reference authority remains public main, including research: Codex thread scope evidence (R64 clean-history candidate) #28 (Thread.environments selection/runtime diagnostic) and research: separate workspace roots from write authority #33 (runtimeWorkspaceRoots != writableRoots);
• draft research: bind write permission evidence to trusted source #34 was reconciled to the current clean base and its stale privacy blocker text was removed: base eee8cd79148430bbac57148152e691ec8f87aa2a, head 3db8cf123a428a5e93345c2db9ddac02e619cfe0, Scope Binding Reference 33924090723 SUCCESS with 44/44 tests, Public privacy 33924090724 SUCCESS, Vercel SUCCESS; research: bind write permission evidence to trusted source #34 remains DRAFT / NOT MERGED;
• the existing protocol-level comment on openai/codex#24224 was updated in place (no new upstream comment) to use the canonical RUMBO-IA/Rumbo reference and current research: bind write permission evidence to trusted source #34 receipt;
• no claim of OpenAI endorsement, root-cause proof, or upstream fix; no automatic merge or production promotion.

Separate repository hygiene hardening is tracked in #38 (fix(privacy): scan metadata across fetched public refs), currently OPEN / NOT MERGED. This is preventative and does not alter the agent-reliability acceptance model.