Wayfinder: Spot, Perps and Predictions MCP evaluations

[ericjuta]

Current checkpoint, 2026-09-11

Canonical #73 implementation and admission checkpoint. PR #79 and PR #80 are merged, superseding the obsolete runner-implementation blockers. Source landing and synthetic proof are not venue admission or measured acceptance.

Dedicated venue execution evidence remains incomplete. Suitable real authentication/profile access, Gina MCP authorization, routing/reasoning evidence, concrete campaign bounds, UTC expiry, run approval and separate exact-content publication approval remain open. #73 stays open; #71 stays open and blocked by #73. M1-M16, all four profiles, all 34 goals, controlled Sol/medium and Spot-first sequencing remain settled.

Destination

A decision-complete evaluation specification for the dedicated Spot, Perps and Predictions MCPs. It must define two separate layers: deterministic MCP contract/safety checks and natural-language agent task evaluations, with domain coverage, grading evidence, safe execution boundaries, admitted runner profiles and acceptance criteria clear enough for implementation.

Notes

Confirmed scope

The owner selected all three dedicated MCPs, including research, account, trading and automation capabilities, and selected layered coverage plus agent quality. This is a planning map, not permission to implement, access accounts, execute transactions, run paid models or publish measured results. Covering a write intent does not authorize performing that write.

Use the public documentation index, Spot MCP features, Perps MCP features, Predictions MCP features and public askgina/plugins contracts as published references. Public notes must omit private source identities, paths, implementation details, credentials, account identifiers and raw captures. Document missing public evidence rather than presenting internal facts as publicly established.

Reuse, do not reopen

• Multi-provider and native-agent evaluations owns the existing runner/authentication/comparison program. Its source implementation is merged; it is not dedicated venue evaluation coverage.
• Reuse the approved local comparison conditions: fixed controlled-model conditions, separately identified native-agent evidence, and separate task/activation claims. The existing four-case Gina Read smoke is not a venue acceptance matrix.
• Authentication and isolation policy is confirmed. Versioned run configuration and evidence identity is owner-confirmed; exact native executable pins and upstream endpoint slugs return for approval during runner admission. This map links the owning decisions; policy approval is not runtime admission.
• The retained intermittent OMP MCP-error failure remains unresolved. Its bounded investigation is complete; this map does not reopen it. OMP provider retries remain documented native behavior, not a one-request guarantee.
• Existing exact-report-bound capture, private evidence and explicit manual publication approval remain in force. Synthetic, recorded-response and live evidence must not be conflated.

Planning terms

A venue MCP is one dedicated Spot, Perps or Predictions connection, not the combined Gina Read connection. A contract/safety check evaluates a declared tool/workflow invariant against controlled evidence. An agent task trial evaluates a natural-language user goal under an identified connection, account state and runner profile. Those are separate evidence layers, not interchangeable scores.

The evaluation vocabulary records these terms on the isolated research branch. It is a glossary, not a resolved grading or execution policy.

Decisions so far

• Venue runner policy agreement: owner-confirmed profile/tool conditions and first-live task-rejection policy; numeric action caps return with concrete case proposals. Runner admission remains open for exact runtime/routing evidence and owner approval; measured acceptance remains blocked.

• Cross-provider result identity and evidence policy: owner-confirmed model, reasoning, budget, package, routing and request-setting rules; native executable pins and exact upstream endpoint slugs return during runner admission for owner approval before credential-bearing execution. Policy closure is not runner admission or measured proof.

• Spot workflow research: 23 native tools plus MCP bash, explicit intent/settlement distinctions, and candidate coverage with unresolved schemas and safety oracles.

• Perps workflow research: venue-scoped bash workflows require identity and outcome evidence; public feature examples do not establish complete trading, funding or SQL receipts.

• Predictions workflow research: event/outcome/time identity, execution state and analytics/scheduling evidence must be graded separately; public gaps remain explicit.

• Reusable eval evidence research: current Gina Read routing/argument/completion contracts are reusable, but nested venue actions, result grounding, authorization and state outcomes need new admitted evidence.

• Safe venue execution and account-state boundaries: owner-confirmed bounded, attended live trading in dedicated accounts; concrete campaign approval and separate runner-admission and measured-acceptance decisions remain required.

• Separate contract and agent-task grading oracles: owner-confirmed outcome and evidence rules, durable started-trial accounting and Fail / Inconclusive / Unavailable / Pass precedence; contract, task and activation claims stay separate.

Venue research, safety and grading decisions are resolved. The shared authentication and identity policies are also resolved; runner admission and measured acceptance remain open.

Not yet specified

• Additional adversarial, recovery and long-running workflow cases that emerge from the first domain/evidence inventories.
• Statistical expansion, refresh cadence and historical comparability after representative cases and grading oracles are chosen.
• The final implementation sequence once the decision frontier is resolved.

Open questions already expressed as child tickets live in those tickets, not a second checklist here.

Out of scope

• Implementing new runners, changing MCP services, running model/MCP evaluations, provisioning credentials or touching live funds while charting.
• Reopening completed native-runner diagnostics, forking OMP to change retries, or silently replacing accepted comparison/auth decisions.
• Trading profitability, investment advice quality or financial-performance claims as a substitute for task correctness and safety.
• Automatic approval of public results, production rollout, consumer CI credential distribution or repository release work.

[ericjuta]

Published admission research checkpoint

Includes all 16 settled decisions and all 34 candidate goals, with their goals, categories, oracles, and proposed modes: 17 read/no-action, 11 live-financial, and 6 synthetic-provider-fixture agent tasks. Separate contract coverage and retained families remain in scope. Counts, costs, routes, accounts, and executable inputs are not frozen or approved.

Both approval assessments remain NOT_READY_FOR_APPROVAL. Runner admission stays open; Campaign acceptance remains blocked by it. This map stays open. Sanitized documentation publication does not authorize execution or measured-results publication.

[ericjuta]

Source checkpoint, 2026-09-10: public evaluator 99f6fd1 has passing local and CI proof. The source and admission checkpoint records the supported claims. Dedicated venue execution evidence remains incomplete; this is not venue admission or measured acceptance.

This map stays open. All locked decisions, four required profiles, all 34 candidate goals, and controlled Sol/medium with Spot first remain unchanged. The case matrix and measured acceptance gate stays blocked by runner admission. No campaign or measured-publication approval is inferred.

[ericjuta]

Updated source preparation checkpoint: evaluator 0f554b8 is published with 324 passing tests and exact-head CI/CodeQL. Source and offline proof only; unmerged and NOT_READY_FOR_APPROVAL. Required admission evidence and campaign approval remain outstanding. Locked decisions and dependencies are unchanged; #73 stays open and #71 stays blocked.

[ericjuta]

Restricted Codex adapter source is published at 77f2a75, with 359 tests, package-install proof, and exact-head CI/CodeQL. It remains unmerged. Canonical source proof and remaining admission gates. Admission remains NOT_READY_FOR_APPROVAL; #73 stays open and #71 remains blocked. All four profiles, 34 goals, Sol/medium and Spot-first sequencing are unchanged. This is source/offline proof, not live/native or venue acceptance and not campaign approval.

[ericjuta]

PR #77 is merged into main as d0bff12; main CI/CodeQL passed. Canonical landing evidence and remaining admission gates. Source landing does not enable native credential-bearing evaluation or approve a campaign. NOT_READY_FOR_APPROVAL remains, #73 stays open and #71 remains blocked. Required coverage and sequencing are unchanged.

[ericjuta]

New account-free native findings: native filesystem/network enforcement now runs, but the current Unix relay is blocked by Linux seccomp; all-process existing-login custody also remains unproven. No credential-bearing evaluation ran. Source stays merged at d0bff12; admission remains NOT_READY_FOR_APPROVAL and measured acceptance remains blocked. Locked decisions, coverage and dependencies are unchanged.