Skip to content

Fix flaky test on site theming mcp tool#222

Merged
sf-cboscenco merged 1 commit intomainfrom
fix-site-theming-mcp-tool-flaky-test
Feb 27, 2026
Merged

Fix flaky test on site theming mcp tool#222
sf-cboscenco merged 1 commit intomainfrom
fix-site-theming-mcp-tool-flaky-test

Conversation

@sf-cboscenco
Copy link
Contributor

Summary

Fix flaky test on site theming mcp tool

Testing

This is a test fix.

Dependencies

  • No net-new third-party dependencies were added
  • If net-new third-party dependencies were added, rationale/discussion is included and 3pl-approved is set by a maintainer

  • Tests pass (pnpm test)
  • Code is formatted (pnpm run format)

@sf-cboscenco
Copy link
Contributor Author

Root cause: flaky testDir and early return in initialize
The failures come from two things:

  1. testDir can be identical across tests
    testDir is created with:
    theming-store.test.ts
    Lines 23-23
    testDir = path.join(tmpdir(), b2c-theming-store-test-${Date.now()});
    Date.now() only changes every millisecond. When tests run quickly, several can share the same testDir.
  2. initialize skips when the root is unchanged
    In theming-store.ts:
    theming-store.ts
    Lines 364-366
    if (this.initializedForRoot === root) { return; }
    If two tests get the same testDir, the second call to initialize(testDir) returns immediately and never processes the new THEMING_FILES from the current test.
    How this breaks the failing tests
    "should resolve absolute paths in THEMING_FILES"
    Sets THEMING_FILES and calls initialize(testDir).
    If testDir matches the previous test, initialize returns early and never loads the custom file.
    has('absolute-theming') stays false.
    "should clear and re-load when root changes"
    Same pattern: if testDir collides with another test, initialize may not run as expected and second-root is never loaded.
    Evidence
    Running the suite 20 times showed intermittent failures (sometimes 1, sometimes 2), which matches this timing-based collision.
    Fix
    Make testDir unique per test so initialize always runs. For example:
    testDir = path.join(tmpdir(), b2c-theming-store-test-${Date.now()}-${Math.random().toString(36).slice(2, 9)});
    Or use a counter:
    let testCounter = 0;// in beforeEach:testDir = path.join(tmpdir(), b2c-theming-store-test-${Date.now()}-${++testCounter});
    Either approach prevents testDir collisions and removes the flakiness. I can switch to Agent mode and apply this change if you want.

@sf-cboscenco sf-cboscenco merged commit 362c935 into main Feb 27, 2026
4 checks passed
@sf-cboscenco sf-cboscenco deleted the fix-site-theming-mcp-tool-flaky-test branch February 27, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants