Unit testing Linaria-based components

[kurtdoherty]

While implementing #150 and #151, I ran up against some unexpected constraints in the unit tests for TextArea when using React Testing Library (RTL) to check whether the component behaved as I expected.

1. RTL could not find the <textarea> element the component renders via screen.getByRole('textbox');
2. RTL could not assert that the "shadow" text area (which is a visually hidden text area element use to help with the JS-based resizing logic) was hidden (not visible).

Investigating these a bit further:

• (1) is caused by our global mocks for Linaria's styled and css utilities. It appears as though we can remove these safely, though we'll need update every single snapshot because the HTML tags and CSS class names captured will no longer have the mock-styled. or mock-css. prefixes. 🥳
• (2) is due to Linaria needing a compile-time step to extract the actual CSS styles for each class. This compilation work is not done by Jest, so when we render components with RTL, they're not rendered with "real" classes and thus, have none of the styling we might expect when we place it under test.

So (1) is easy to solve, but (2) does not appear as straightforward. I haven't been able to find much documentation around testing with Linaria, so it's hard to know if there's something simple I'm missing or not.

Why is this a problem? Personally, I want to be able to sufficiently test shared UI components without resorting to browser-based tests, and at the moment, it seems like we can't do that with Linaria 😬 To give a concrete (though contrived) example:

// test-button.tsx
type TestButtonProps = {
  children?: ReactNode
  isVisuallyHidden?: boolean
}

const TestButton = styled.button<TestButtonProps>`
  visibility: ${({ isVisuallyHidden }) => {
    return isVisuallyHidden ? 'hidden' : 'visible'
  }};
`

// test-button.test.tsx
test('can be visually hidden', () => {
  render(<TestButton isVisuallyHidden>Foo</TestButton>)
  expect(screen.getByRole('button', { name: 'Foo' })).not.toBeVisible()
})

Running the above unit test will result in the following failure. You can see that our <button> is correctly rendered (with issue 1 above fixed locally), but the class attribute is a fake class name, not the real one and thus, the dynamic styles are not being applied and we cannot assert the correct behaviour of isVisuallyHidden 😞

I'm very interested to know if anyone else is concerned about what this means for our ability to unit test component behaviour that cannot be captured via simplistic DOM snapshots. I suspect this is probably a small part of a larger conversation around testing practices in Elements and what kind of coverage we expect to have for our components.

[kurtdoherty]

note: I'm hoping to spend some time looking into whether we can get Jest performing the necessary CSS extraction as it runs tests or not. If not, I'll look at whether it's easier to do with Vite/Vitest.

[kurtdoherty]

I've been able to get it working with Vite + Vitest, but not Jest, though I find Jest much harder to configure than Vitest, so maybe it's still possible. With Vitest, since it uses Vite to process each test file, we kind of get all the behaviour we need out of the box.

A quick 5 minute POC let me achieve the testing of the new TextArea (see #150 and #151) that I would have expected to be able to get via Jest. In particular, the shadow text area is visually hidden and is always accessible via the textbox role were not achievable because of the issues mention in the OP.

I'm not sure if we're open to switching from Jest to Vitest though...

[willmcvay]

Thanks for looking into this @kurtdoherty and sorry for the slow reply!

Firstly, kudos on getting this working, really nice work.

I did try out Vitest on Elements and other projects when we migrated from Webpack but at the time, there were some gnarly issues I couldn't resolve so left the Jest config as it was. It looks like Vitest is a lot more mature now so assuming there are no other issues, I am very supportive of moving away from Jest. Just a few thoughts on that generally for the wider room;

• Vitest is a much faster runner so we should get faster pipelines and better devex
• Makes sense that if we are committed to Vite as a build tool, we would also use it to build our tests
• Jest has massively slowed its release cycle since it moved away from being a Meta open source project (last release Sept '23) - it's very stable and a solid choice still but we aren't likely to see huge improvements going forward. Feels in maintenance mode.
• There are limitations as per above when working with Jest and Linaria - if Vitest solves these, it's a no brainer.

If you want to push ahead with making Vitest the runner for the v5 refactor, am in favour. We'd just need to update the contribution guide accordingly and probably provide some guidance on migrating a pre-existing Jest test to Vitest - this should be pretty straigtforward.

Final point, while we migrate if that is the decision, we just need to make sure we can run both suites concurrently ie both the legacy suite and the vitest suite. Unless there is some codemod we can run to do the migration for us? Would be great if so!

[kurtdoherty]

@willmcvay Keen to hear your thoughts on the above when you have a chance.

I notice that we currently rely on:

• Vite for Storybook
• Rollup for the production build
• Jest for tests

All of which involve their own configs. Not sure if this is just some technical debt in Elements or not; are we wanting to consolidate on a single toolchain for builds across Storybook, production assets and our tests?

[willmcvay]

@kurtdoherty I've covered the Jest point above - it's purely a historical decision that seems is not relevant now.

On Vite vs Rollup - Vite still uses Rollup under the hood AFAIK for prod builds. I guess we could consolidate the config into a single Vite file? Would probably get complicated though as the prod storybook might need it's own config?

Worth pointing out Linaria when it comes down to brass tacks is a Babel plugin that pre-processes the CSS-in-JS syntax. Whichever way we slice it, we'll be applying the same Babel transforms.

If you can see a way to tidy up, am in favour but it's not causing any issues currently I am aware of.

[adrian-reapit]

@kurtdoherty We moved the Go app repo to vitest after some tests showed it was faster. Once we moved the larger sections of code we found noticeably slower performance in Github builds (nearly twice as long) but local test speed is faster. Vitest in github builds does not seem to like 'barrel files' with 'export * from my/dir'. It takes longer to process and a test showed that cleaning dependency structures can help. DOM tests may also be slower. We reverted the biggest chunk of code back to jest to speed up the builds and plan to clean up dependency structures and try Vitest there again. There are numerous articles online about Vitest being slower after being told it was quicker but if Jest is not being actively updated then it seems likely that Jest will be replaced in a similar way to enzyme being replaced by testing-library

[kurtdoherty]

@adrian-reapit thanks for the input. We've been migrating our monorepo for Console Cloud over to Vitest from Jest as well, and have also found that its performance is largely on par with Jest (sometimes its faster, sometimes its slower). We haven't noticed any significant difference between local runs and CI though 🤔

Re barrel files, yes, these can cause lots of grief. They are causing us pain too, especially when they export many unrelated things. I think they're fine if everything exported from the barrel file is likely to be used together (e.g. using a barrel file for a component/set of components would have little impact), but using them, for example, to import components from Elements is probably not ideal.

Re DOM tests, we've seen some benefits to using Happy DOM instead of JSDOM, but I won't be surprised if Happy DOM loses its performance edge as it increases its DOM API support.

I think the main benefits of using Vitest over Jest right now are:

1. It's ESM-first, which an increasing number of third-party libs are becoming;
2. It shares its config with Vite, so it reduces the number of different toolchains a team needs to maintain.

I'm going to write up a short solution design this sprint to outline the work that would be required to move Elements from Jest to Vitest, some of the tradeoffs involved, and some of the other tangential concerns (like the current snapshotting approach). If we're all happy with how the solution design looks, I'm happy to do the work.