[Bug]ReferenceError: __test is not defined

[VictorGlindasPaf]

I seem to get this error seemingly at random when running the tests.
Following the stack trace just takes me to the very bottom of my story file.

 FAIL   browser: chromium  src/Tag.stories.tsx (26.779 s)
  ● Base Components/Tag › Variants › smoke-test

    page.evaluate: ReferenceError: __test is not defined

      at eval (eval at evaluate (:3:2389), <anonymous>:4:17)
      at t.default.evaluate (<anonymous>:3:2412)
      at t.default.<anonymous> (<anonymous>:1:44)
      at testFn (src/Tag.stories.tsx:84:37)
      at Object.<anonymous> (src/Tag.stories.tsx:99:17)

[yannbf]

Hey @VictorGlindasPaf thanks for opening this issue! Is that the only failure in your tests, or do previous tests fail too?
Is there more info you can give to help us reproduce the issue?

[VictorGlindasPaf]

These errors were definitely mixed with other ones, but I'm not sure in which order they came.

I'll see if I have time to get some more info about this when back at work on Monday.

[valentinpalkovic]

@yannbf The error is flaky and happens occasionally. Yesterday, even in this repository, it happened once when I was running the tests. Reexecuting the tests made the error disappear. It seems to be a kind of timing issue. This happens occasionally on my private Gitlab pipeline as well (flaky). Locally it just happens rarely.

[yannbf]

@yannbf The error is flaky and happens occasionally. Yesterday, even in this repository, it happened once when I was running the tests. Reexecuting the tests made the error disappear. It seems to be a kind of timing issue. This happens occasionally on my private Gitlab pipeline as well (flaky). Locally it just happens rarely.

Interesting! Can we chat more about it? Can you reach out on discord at Yann#9096 or twitter?

[valentinpalkovic]

@VictorGlindasPaf I had the same issue. In my case, the issue was a redirection happening in the components by using a decorator, redirecting to an SSO page when a Token was not available in the localstorage. Maybe you have a similar issue and a redirection happens in special cases or you are triggering a page reload. This would explain, why the initial script, which contains the __test function, is not available in the head section anymore.

[janivo]

I'm having the same problem. The error seems to occur at random. I'm using the test-runner to take snapshots of stories of web components.
Because the web components are not hydrated at the time the postRender hook runs, I added a waitForFunction to wait for my components to have the .hydrated class.

This is my test-runner.js file:

module.exports = {
  async postRender(page, context) {

    await page.waitForFunction(
      () => {
        const inoElements = Array.from(
          document.getElementsByTagName('*')
        ).filter((el) => el.tagName.startsWith('INO'));

        return inoElements.every((el) => el.classList.contains('hydrated'));
      },
      { polling: 50 }
    );

    await page.$('#root');

    const elementHandler = await page.$('#root');
    const innerHTML = await elementHandler.innerHTML();
    expect(innerHTML).toMatchSnapshot();
  },
};

The introduction of this snippet led to this error. Any ideas why this is happening?

[ericclemmons]

I can reproduce this by setting resetContextPerTest: true for https://github.com/playwright-community/jest-playwright.

[joriswitteman]

For me, this is caused when I navigate to a different URL trying to capture snapshots during postRender(). Probably the test runner injects something into the page that is lost by navigating to another URL.

(The use case for navigating for me, is to put different arg values in the URL and as such capture different permutations of the Story).

[gmwill934]

getting same error, following the docs, it says zero config, but do we need extra config?

[AnnaHoege]

I'm also getting this error while testing web components.
It started after migrating to Storybook 7 and switching builders from webpack to vite.

Now tests will also fail at random, probably because vite takes too long for the test-runner to render components.