Skip to content

Flaky test: codex/claude adapter skill-injection tests collide on shared tmpdir paths #156

Description

@tadasant

Symptom

CI intermittently fails with:

× CodexAdapter > prepareSession > injects skills into .agents/skills/
  → expected '# Catalog version' to contain '# Deploy'

Observed on PR #155 — failed on test (18) and test (20), passed on test (22), passed on rerun. Not reproducible reliably; passes locally.

Root cause

Vitest runs test files in parallel workers. Two adapter test files write skill-source SKILL.md files to shared, non-unique paths under the OS temp dir:

  • packages/extensions/adapter-codex/tests/codex-adapter.test.tscreateTempDir() makes a unique air-codex-test-<ts>-<rand> dir, but the test then writes its skill source to join(dir, "..", "skills", "deploy") — i.e. a sibling path (<tmpdir>/skills/deploy) that is not unique to the test. It writes # Deploy, then asserts the injected file contains # Deploy.
  • packages/extensions/adapter-claude/tests/claude-adapter.test.ts:542 writes # Catalog version to its own catalog skill source.

When the two files run concurrently and their skill-source paths overlap (shared <tmpdir> ancestry with a fixed skills/deploy leaf), one worker's # Catalog version clobbers the other's # Deploy between write and read, so the codex assertion sees # Catalog version.

Fix direction

Keep every test's skill-source files inside that test's unique temp dir, never in a shared sibling path. E.g. write skill sources under join(dir, "catalog", "skills", "deploy") (a child of the unique temp dir) instead of join(dir, "..", "skills", "deploy"). Audit both adapter test files (and any others using the join(dir, "..", …) pattern) for the same shared-path issue.

Notes

Pre-existing; surfaced on PR #155 but unrelated to that change (which is scoped to plugin manifest resolution). Low severity (rerun clears it) but recurring CI noise.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    convergentIssue-work-gate direction: closes a gap the system is already supposed to have closed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions