Feature request: Shadow DOM support

[apalaniuk]

Describe the feature you'd like:

Web components are an increasingly popular way of creating reusable elements. Due to the existence of shadow DOM, user-focused testing is more difficult and requires tooling focused around the issue, as in the most permissive case, element queries/observer events (eg. via MutationObservers) stop at shadow boundaries and must explicitly be projected into shadow trees. Without support from tooling, aspects of a web app written using web components are virtually untestable without using methods contrary to the goals of this project.

Suggested implementation:

• Standard element queries can be modified to be projected into successive shadow trees
• Mutation observers to support eg. waits for DOM tree changes can be attached to all shadow roots of descendants
• Solution must consider related concepts (eg. slots) insofar as they relate to testing goals and principle of least surprise according to their documented behaviour/DOM standards
• No expectation of closed shadow roots to be supported in any way, but ideally would be explicitly documented

Describe alternatives you've considered:

• As first-class support is required, alternatives involve using/writing an alternative implementation which supports shadow trees explicitly (or potentially, writing a plugin system to allow others to support it)
• As shadow tree concepts are an aspect of WHATWG/W3C DOM standards, it makes sense that they're supported first-class by tooling that supports DOM operations

I'd be happy to add this feature, and support for basic functionality (query, waits) was straightforward to implement, but I wanted to get insight on constraints/concerns in advance as I don't currently use this library.

[ashubham]

+1

It will be great to build support for WebComponent based frameworks like lit-element. If there is a adapter/plugin API, I am sure the community will jump on to build it.

[alexkrolick]

How much of this depends on JSDOM itself? This library doesn't implement the DOM part, just traversal.

[apalaniuk]

Hi @alexkrolick, it's a good question. JSDOM currently has shadow DOM support, with a few restrictions. The largest issue for testing right now is that this is almost entirely used in the context of web components, and support for custom elements is in-flight (jsdom/jsdom#2548).

That said, the only changes required in here for shadow DOM support should be unrelated, since this would just need to support automatically traversing through shadow trees for queries/attaching MutationObservers/etc, and shadow roots can be attached to many standard elements.

Since the overall change seems small, it's probably better to just implement in a fork and get more direct feedback based on an implementation.

[kentcdodds]

A PR would be helpful. But I'm getting the impression this will be anything but a small change 😬

[ashubham]

What about when we use this library with puppeteer ? How difficult will be to have webcomponent support in that mode ?

[apalaniuk]

@kentcdodds There's definitely a lot of complexity to work out, so I'm just being (overly) optimistic 🙂 we have done similar things at my company in the past (though only supporting a subset of the required operations), but supporting it for general use cases will take some thought (complexity to get same behaviour in traversing the light DOM given there aren't any primitives for traversal/element selection when considering shadow DOM, performance, maintainability, providing unsurprising behaviour in all cases, etc.)

[Westbrook]

I've been toying with an approach to making this possible: adobe/spectrum-web-components#420

For me, there are two issues in particular that I'm working on:

1. ES Module based support so that Karma + es-dev-server can run @testing-library/dom which has some downstream CJS dependencies, for which I'm rebuilding the project locally: https://github.com/adobe/spectrum-web-components/pull/420/files#diff-de467343944f9302cd195062f4d866b2 it also knocks off some Node vs browser differences
2. Seeing into shadow DOM appropriately (normally an anti-pattern, but acceptable, IMO, in a testing environment), for which I'm using: https://github.com/adobe/spectrum-web-components/pull/420/files#diff-33d340131f49793bce88e37a2122f57c

At the intersection between the two issues is shimming @testing-library/dom to use the non-standard querySelectorAllWithShadowDOM so that the "available" DOM is queried without altering the API of the components.

Is this looking completely crazy? Are either or both of these changes something that might look interesting in a PR here?

[Westbrook]

So, in a way, the above is also a bit of a revival of #161 as well... 😬

[kentcdodds]

Hi folks,

I just am not convinced that the complexity will be worth it. You can feel free to open a PR if you like, but if it's more than a little complex I don't think I'll want it here and instead maybe you could add a custom query and even make a library of those for the people using Shadow DOM.

[Westbrook]

Definitely the steps for full web component support could be more complex, but ensuring that DOM targetted code can actually be tested in a real DOM environment would be a great baseline for this library to support. Then, were someone to want to extend/alter it to better support the growing capabilities of the full DOM model itself they could so with little difficulty. To that end, I think the initial steps that I'd love to see are as follows:

• Don't rely on process.env: see fix(pretty-dom): Ensure safe process.env access #431
• Don't rely on CJS dependencies: This one is a little more tricky (however seems to be contained to @sheerun/mutationobserver-shim at this time) and it would be better to get some insight on what is deemed acceptable before I move forward.

The best solution is to PR the library to export ESM on its own (if you were willing to support an issue/PR of that there, the voice of such a well-used and respected library would likely go a long way, so I'd be happy to open the conversation for you to jump in there), but as we've seen not everyone is excited about supporting modern APIs in their libraries. In the interim of that occuring, I'd like to pre-process @sheerun/mutationobserver-shim via an addition to the Rollup config similar to (output locations TBD, etc, etc):

    {
        input: require.resolve(
            '@sheerun/mutationobserver-shim/MutationObserver.js'
        ),
        output: {
            file: './lib/mutationobserver-shim/index.js',
            format: 'es',
        },
        plugins: [
            resolve({
                browser: true,
                preferBuiltins: false,
            }),
            commonjs(),
        ],
    },

Then it's a simple alias to slot this new output into @test-library/dom without having to affect the current contents of the JS (meaning it's plug&play when the upstream output changes!). This looks like:

            alias({
                entries: {
                    '@sheerun/mutationobserver-shim':
                        'lib/mutationobserver-shim/index.js',
                },
            }),

With these two changes, it is likely that #161 could be reopened/closed as fixed in one fell swoop and then we could have a stand-alone conversation about supporting the web component specifications. The decision as to whether that could be achieved via custom queries or should belong in @test-library/dom, in an alternate @test-library/full-dom sort of package, or be supported via an external extension of the library is an important one and should be given the focused consideration that it deserves.

Thanks in advance for your gracious insight.