Specify Codex marketplace conformance and install verification

[ericjuta]

Part of #19

Decision output

Specify the permanent checks that distinguish a valid repository marketplace from a merely discoverable legacy marketplace file.

The contract must cover:

• canonical .agents/plugins/marketplace.json schema, required policy fields, Finance category, and source containment;
• source-root loadability: .codex-plugin/plugin.json, declared canonical skills, .mcp.json, and assets all resolve inside the selected root;
• equality of marketplace, manifest, and generated OpenAI artifact identity/version where selected;
• preservation of strict generated-host artifact isolation and existing archive-security limits;
• an actual Codex marketplace add/install smoke from an isolated clean checkout without mutating user-global state;
• deterministic CI-safe coverage versus explicitly local/manual proof.

Acceptance

The resolution names exact checks, commands, fixtures, cleanup boundaries, and observable failures. Record it in a resolution comment, close the ticket, and append only a gist plus link to #19 Decisions so far.

Non-goals

Do not implement checks or perform production/user installation.

[ericjuta]

Prototype #21 confirmed zero-build Codex install and root extras tolerance. It also observed that ON_INSTALL remained listing metadata but Codex CLI did not initiate OAuth or contact Ask Gina. This ticket must specify the expected auth proof and the manual ChatGPT check. Evidence: https://github.com/askgina/plugins/blob/5b57071738e51dbf12dd1c645d5adbd37cc46793/PROTOTYPE_CODEX_MARKETPLACE.md

[ericjuta]

Resolution: Codex Marketplace Conformance and Install Verification Contract

Verdict: Confirmed & implementation-ready.

Based on locked distribution contract #22, prototype installation evidence in #21 (commit 5b57071738e51dbf12dd1c645d5adbd37cc46793), and migration contract #20, this specification defines the permanent static conformance checks, isolated automated Codex remote smoke verification, authentication boundary semantics, and manual ChatGPT desktop validation protocols for repository marketplace distribution.

---

1. Canonical Marketplace Schema, Policy, and Source Containment

A. Schema & Policy Specification (.agents/plugins/marketplace.json)

The repository root marketplace descriptor must strictly validate against the following contract:

{
  "name": "ask-gina-plugins",
  "interface": {
    "displayName": "Ask Gina Plugins"
  },
  "plugins": [
    {
      "name": "ask-gina",
      "source": {
        "source": "local",
        "path": "./plugins/ask-gina"
      },
      "policy": {
        "installation": "AVAILABLE",
        "authentication": "ON_INSTALL"
      },
      "category": "Finance"
    }
  ]
}

B. Static Conformance Assertions (tools/check-target-conformance.ts)

The conformance checker must assert:

1. Root Marketplace Structure:
   • name matches ask-gina-plugins.
   • interface.displayName matches "Ask Gina Plugins".
   • plugins is a 1-element array.
2. Plugin Entry Contract:
   • name === "ask-gina".
   • source.source === "local".
   • source.path === "./plugins/ask-gina".
   • policy.installation === "AVAILABLE".
   • policy.authentication === "ON_INSTALL".
   • category === "Finance".
3. Source Containment & Boundary Proof:
   • source.path resolves strictly within the repository workspace (rejects .. escapes or symlinks leading outside repository boundary).
   • Resolved directory plugins/ask-gina/ is a valid directory containing the directly loadable plugin source.
4. Claude Marketplace Parity:
   • .claude-plugin/marketplace.json continues to validate independently under existing Claude schemas.

---

2. Source-Root Manifest, MCP, Assets, and Skills Resolution

The canonical source root at plugins/ask-gina/ must resolve all declared assets and dependencies locally with zero build or compilation requirements:

1. Manifest Integrity (plugins/ask-gina/.codex-plugin/plugin.json):
   • name: "ask-gina".
   • version: Matches RELEASE_VERSION from @askgina/contracts.
   • skills: "./skills/" -> strictly resolves to plugins/ask-gina/skills/ containing all 4 canonical skills (research-hyperliquid, research-prediction-markets, research-spot-tokens, review-gina-account), each containing valid SKILL.md and agents/openai.yaml.
   • mcpServers: "./.mcp.json" -> strictly resolves to plugins/ask-gina/.mcp.json.
   • interface.composerIcon: "./assets/icon.svg" (exists and is a valid SVG).
   • interface.logo: "./assets/icon.svg" (exists and is a valid SVG).
   • interface.category: "Finance".
   • interface.supportURL: "https://askgina.ai/support".
2. MCP Binding (plugins/ask-gina/.mcp.json):
   • Contains single server key "ask-gina".
   • type: "http".
   • url: "https://askgina.ai/ai/gina/mcp".
3. Clean Overlay Cutover:
   • Asserts absence of legacy plugins/ask-gina/targets/openai/.
   • Asserts no symlinks exist in plugins/ask-gina/.

---

3. Single Unified Version Equality Invariant

The verification pipeline (tools/verify-artifacts.ts and tools/check-target-conformance.ts) enforces strict semantic version equality across all repository manifests and artifacts:

• Root package.json (version)
• Workspace packages: packages/contracts/package.json, packages/sdk/package.json, packages/cli/package.json, packages/evals/package.json, plugins/ask-gina/package.json
• Release constant RELEASE_VERSION in @askgina/contracts
• Canonical OpenAI manifest plugins/ask-gina/.codex-plugin/plugin.json (version)
• Claude manifest plugins/ask-gina/targets/claude/.claude-plugin/plugin.json (version)
• Claude marketplace .claude-plugin/marketplace.json (version and plugins[0].version)
• Generated target archives dist/targets/ask-gina-${host}-${version}.tgz
• Signed artifact receipts dist/receipts/contract.json and dist/receipts/packages.json

Any version mismatch fails verification immediately with ArtifactVerificationError.

---

4. Generated Public Host Artifact Isolation & Archive Security

Even though the repository marketplace source plugins/ask-gina/ contains ordinary workspace files (e.g. package.json, plugin.yaml, README.md, LICENSE, src/, evals/, targets/, __tests__/), the generated public archive dist/targets/ask-gina-openai-${version}.tgz must remain strictly lean:

1. Lean File Set Invariant:
   • Extracted archive contains strictly:
     • .codex-plugin/plugin.json
     • .mcp.json
     • assets/icon.svg
     • skills/<skill_name>/SKILL.md (4 skills)
     • skills/<skill_name>/agents/openai.yaml (4 skills)
   • Contains zero foreign host files, tests, package manifests, or source code.
2. Deterministic Archive Packaging (tools/pack-artifacts.ts):
   • Created via tar with flags --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner -czf.
   • Preserves staged permissions while normalizing mtime, ownership, and entry ordering.
3. Existing Archive Security Controls (tools/archive-security.ts):
   • Enforces DEFAULT_ARCHIVE_LIMITS:
     • maxCompressedBytes: 64 MiB (67,108,864 bytes)
     • maxFileBytes: 16 MiB (16,777,216 bytes)
     • maxExpandedBytes: 128 MiB (134,217,728 bytes)
     • maxMembers: 10,000 members
     • maxDepth: 32 path segments
     • maxPathLength: 512 bytes
     • maxTimestampSeconds: 4,102,444,800 (year 2100)
   • Structural & format verification:
     • Strict POSIX ustar magic and header checksum validation.
     • Header parsing of octal mode, uid, gid, size, and timestamp fields.
     • Rejection of symbolic links and hard links via member link target rejection (fieldBytes(header, 157, 100) must be empty).
     • Strict path normalization: rejects directory traversal (..), leading slashes, Windows drive prefixes, backslashes, control characters, and duplicate/conflicting paths.
4. Public Boundary Secret Scans (tools/check-public-boundary.ts):
   • Scans all public files and receipts against high-confidence secret patterns (tokens, keys, userinfo) and private repository keywords.

---

5. Automated Isolated Codex Remote Smoke Contract (Deterministic CI)

A. Hermetic Harness & Zero-Global-State Guarantee

The automated smoke test must run hermetically without touching or mutating developer ~/.codex/ or system configuration:

• Creates an ephemeral temporary directory TMPDIR=$(mktemp -d).
• Sets isolated environment variables: HOME="$TMPDIR/home", XDG_CONFIG_HOME="$TMPDIR/config", XDG_CACHE_HOME="$TMPDIR/cache".
• Uses deterministic trap/acquire-release cleanup (trap 'rm -rf "$TMPDIR"' EXIT INT TERM) to purge the sandbox upon test completion or error.

B. Smoke Execution Lifecycle (Remote/Ref Syntax Proven in #21)

# 1. Add remote repository marketplace by owner/repo and ref
codex plugin marketplace add askgina/plugins --ref <git-ref> --json
# Assert: output resolves marketplace name "ask-gina-plugins" and matches checkout commit SHA

# 2. List available plugins in the marketplace
codex plugin list --marketplace ask-gina-plugins --available --json
# Assert: output contains ask-gina@ask-gina-plugins, version matches RELEASE_VERSION,
# source "plugins/ask-gina", policy { "installation": "AVAILABLE", "authentication": "ON_INSTALL" }, category "Finance"

# 3. Install plugin from marketplace
codex plugin add ask-gina@ask-gina-plugins --json
# Assert: exit code 0; cache directory created at $HOME/.codex/plugins/cache/ask-gina-plugins/ask-gina/<version>/

# 4. Verify installed cache contents
# Assert: .codex-plugin/plugin.json, .mcp.json, assets/icon.svg, and 4 canonical skills exist in the cache directory

# 5. Remove plugin and marketplace (exercised during cleanup)
codex plugin remove ask-gina@ask-gina-plugins --json
codex plugin marketplace remove ask-gina-plugins --json
# Assert: uninstallation and marketplace deregistration succeed with exit code 0

C. Deterministic CI Safety

• Requires zero API keys, production tokens, or external auth credentials.
• Does not perform live network calls to OpenAI backends during local file inspection.
• Executes non-interactively in standard CI runners (bun run check:marketplace:codex or bun test).

---

6. Authentication Boundary: ON_INSTALL Metadata vs Live Connection Handshake

Prototype #21 proved that CLI codex plugin add records "authentication": "ON_INSTALL" as listing metadata but does not trigger interactive OAuth or contact Ask Gina endpoints during CLI install. The verification contract explicitly defines the boundaries:

1. Marketplace Metadata Level (CI-Safe):
   • Validates that .agents/plugins/marketplace.json declares "policy": { "authentication": "ON_INSTALL" }.
   • Validates that .mcp.json declares the valid production endpoint (https://askgina.ai/ai/gina/mcp, type: "http").
   • Validates zero-build file registration and manifest schema conformance.
   • Boundary: CI does NOT require or expect live OAuth / network connections during CLI smoke.
2. Live MCP Protocol Preflight Level (packages/evals/src/bin/live.ts):
   • Invoked only during live evaluation runs (bun run evals:live) when explicit credentials and network connectivity are provided.
   • Performs JSON-RPC handshake with https://askgina.ai/ai/gina/mcp (tools/list), asserting advertised tools match @askgina/contracts.
3. Interactive Host Connection Level (ChatGPT Desktop UI):
   • Per official OpenAI documentation, authentication: "ON_INSTALL" indicates that host clients prompt for authentication/connection upon plugin installation or enablement.
   • Verified via the manual pre-release protocol described in Section 7.

---

7. Manual ChatGPT Desktop Pre-Release Verification Protocol

A. Verification Trigger Cadence

• Mandatory prior to merging the initial repository marketplace change (Wayfinder: Codex repository marketplace distribution #19).
• Mandatory on subsequent minor/major releases or modifications to .codex-plugin/, .mcp.json, or OAuth scopes.

B. Step-by-Step Desktop Verification Procedure

1. Marketplace / Plugin Import:
   • In ChatGPT Desktop (macOS/Windows, developer mode enabled), import repository marketplace from local checkout or Git ref.
   • Verify plugin card in the catalog: Display Name ("Ask Gina"), icon rendered, category ("Finance"), description, and "Install" button.
2. Installation & Connection Flow:
   • Click "Install" in ChatGPT Desktop.
   • Observe host connection prompt per OpenAI marketplace standard for ON_INSTALL plugins.
   • Complete authorization flow; verify the plugin successfully installs and reports connected status in the client.
3. Skill & Tool Execution:
   • Start a fresh ChatGPT conversation with Ask Gina enabled.
   • Prompt with an Ask Gina skill query (e.g. portfolio review or token market research).
   • Assert that the plugin executes the tool call and returns valid data in the conversation.

C. Pre-Release Evidence Standard

Manual verification evidence must be recorded in the release PR description or logged to docs/release-evidence/YYYY-MM-DD-v<version>-chatgpt-desktop.md with:

• App build number, platform, and OS version.
• Tested Git commit SHA.
• Screenshots of the install/connection prompt and successful installed state.
• Conversation transcript proving successful skill/tool execution.
• Tester username and timestamp.

---

8. Verification Command Matrix & Diagnostic Failures

Verification Stage	Command Surface	Failure Class / Diagnostic	Scope
Static Marketplace & Target Conformance	bun run check:target-conformance	TargetConformanceError: schema mismatch, missing keys, path escape	Fast CI
Archive Security & Lean Target Isolation	bun run verify:artifacts	ArtifactVerificationError: foreign files in archive, size/count violation	Fast CI
Public Boundary & Secret Scanning	bun run check:public-boundary	PublicBoundaryError: secret tokens or private paths detected	Fast CI
Isolated Automated Codex Smoke	bun test / packages/evals	CodexSmokeError: CLI marketplace add/list/install/remove failure	Isolated CI
Live MCP Handshake (Opt-in)	bun run evals:live	LiveEvalCliError: MCP handshake failed, tool mismatch	Live Evals
ChatGPT Desktop Host Connection Smoke	Manual Desktop Protocol	Interactive failure report with transcript & screenshots	Pre-Release Signoff

---

9. Non-Goals & Boundaries

• This specification establishes verification contracts only; it does not implement checks or perform production installations.
• Authorization for OpenAI universal directory submission remains out of scope.

[ericjuta]

Delivery contract update (2026-08-26): the repository remains private. The delivery owner authorized least-privilege GitHub authentication for the remote Codex marketplace clone after PR #28 proved the credential-free smoke cannot clone the private repository. This supersedes only the credential-free clause in this ticket and #22. The immutable owner/repo + ref proof, isolated HOME/XDG state, pre-install credential deletion, no plugin/runtime credential access, cleanup, and all other conformance requirements remain in force. The canonical implementation plan records the updated boundary in specs/codex-repository-marketplace-plan.md.

[ericjuta]

Initial-release signoff update for PR #28 (2026-08-26): the delivery owner explicitly waived the mandatory ChatGPT Desktop pre-release check for commit e2d468c17fdf2f5f9ff01a57ffd933fba24cd99c and authorized merge based on the green static, artifact, public-boundary, security, external-review, and authenticated Codex remote-install evidence. This waiver means the release does not claim proof of ChatGPT Desktop installation, OAuth connection, or live tool execution. The normal desktop protocol remains applicable to later relevant host-surface changes unless separately waived.