Skip to content

Chromatic publish fails in CI (TeamCity) but succeeds locally #267

@rasinbo

Description

@rasinbo

When running chromatic --playwright locally, everything works perfectly: stories are found, Playwright tests are executed, and the suite is published without issues.
However, when running the exact same command inside our CI environment (TeamCity), the process fails consistently during the step:

Image

This is the log info:

Image
"errorMessage": "✖ Failed to publish your built test suite", 14:19:58 "build": { 14:19:58 "id": "685e8c28274fde3bd629af7f", 14:19:58 "number": 49 14:19:58 }

In some cases it gets past that step successfully, but then fails later during the following stage:

Image

It seems the test suite is either not built correctly in CI, or something modifies files in the background during upload. Additionally, we noticed that in many cases, no stories are actually generated during the process. Even though npx playwright test runs successfully, the generated chromatic-archive folder contains test metadata files with an empty stories array: {"title":"features/chromatic.feature/Chromatic/Darstellung CTA-Button Dark-Mode","stories":[]}.
This suggests that something fails silently when Chromatic attempts to extract or associate stories from the built Storybook. Locally this works as expected, stories are extracted and rendered properly, but in CI the resulting archive is often incomplete or empty in terms of story data.
It’s also worth noting that the issue is difficult to reproduce consistently, since it only occurs within our TeamCity CI environment. Locally everything works fine, and we don’t see any errors when running the same commands.
Because TeamCity runs with its own configuration, environment variables, file system structure, and possible parallel processes, it's hard to isolate exactly what causes the failure. Unfortunately, this makes it difficult for us to create a minimal reproduction outside of CI.

Expected behavior
• The build should succeed like it does locally
• Storybook stories should be found and published
• Files should not be modified mid-upload

Observed behavior in CI
• The following file fails to upload during publish (although in other builds, different files may fail):
/opt/teamcity-agent/BuildAgent/temp/buildTmp/chromatic--66833-*/project.json
• The file reportedly changes during the upload, even though no process should modify it
• Sometimes no stories are detected at all, even though Playwright tests run fine

Additional context
• The issue only happens in CI, never locally
• We run npx playwright test in one step, followed by npx chromatic --playwright in the next
• Occasionally, the build succeeds once, but fails again on the next run with no change in code
• We've confirmed that stories are not generated in CI during Chromatic's build process, even though it works fine locally
• Our TeamCity CI environment uses an outbound proxy that restricts external traffic via URL whitelisting
• We have explicitly whitelisted *.chromatic.com in the proxy configuration to allow Chromatic-related traffic

Environment
CI: TeamCity (Linux agent)
Chromatic : ^12.2.0
Chromatic-com/playwright: 0.12.5
Playwright version: 1.52

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions