Off-screen/hidden elements still queryable

[adiun]

• dom-testing-library version: 3.16.4
• react version: 16.8.0-alpha.1
• node version: 8.11.3
• npm (or yarn) version: 5.6.0

Relevant code or config:

test("off-screen element not queryable", () => {
  const { queryByText } = render(
    <div style={{ height: "150px", overflow: "auto" }}>
      <div style={{ height: "100px" }}>Paragraph-1</div>
      <div style={{ height: "100px" }}>Paragraph-2</div>
      <div style={{ height: "100px" }}>Paragraph-3</div>
      <div style={{ height: "100px" }}>Paragraph-4</div>
    </div>
  );
  const paragraph = queryByText("Paragraph-4");
  expect(paragraph).toBeNull();
});

test("non-displayed element not queryable", () => {
  const { queryByText } = render(
    <div style={{ height: "200px", overflow: "auto" }}>
      <div style={{ height: "100px" }}>Paragraph-1</div>
      <div style={{ height: "100px", display: "none" }}>
        NotDisplayed-Paragraph
      </div>
    </div>
  );
  const paragraph = queryByText("NotDisplayed-Paragraph");
  expect(paragraph).toBeNull();
});

test("hidden element not queryable", () => {
  const { queryByText } = render(
    <div style={{ height: "200px", overflow: "auto" }}>
      <div style={{ height: "100px" }}>Paragraph-1</div>
      <div style={{ height: "100px", visibility: "hidden" }}>
        Hidden-Paragraph
      </div>
    </div>
  );
  const paragraph = queryByText("Hidden-Paragraph");
  expect(paragraph).toBeNull();
});

What you did:

Tried to write the tests above, expected them to pass.

What happened:

Tests failed:

off-screen element not queryable
2ms
expect(received).toBeNull()

Expected value to be null, instead received
  <div style="height: 100px;">Paragraph-4</div>

  14 |     </div>
  15 |   );
  16 |   const paragraph = queryByText("Paragraph-4");
> 17 |   expect(paragraph).toBeNull();
  18 | });
  20 | test("non-displayed element not queryable", () => {
non-displayed element not queryable
1ms
expect(received).toBeNull()

Expected value to be null, instead received
  <div style="height: 100px; display: none;">NotDisplayed-Paragraph</div>

  27 |     </div>
  28 |   );
  29 |   const paragraph = queryByText("NotDisplayed-Paragraph");
> 30 |   expect(paragraph).toBeNull();
  31 | });
  33 | test("hidden element not queryable", () => {
hidden element not queryable
1ms
expect(received).toBeNull()

Expected value to be null, instead received
  <div style="height: 100px; visibility: hidden;">Hidden-Paragraph</div>

  40 |     </div>
  41 |   );
  42 |   const paragraph = queryByText("Hidden-Paragraph");
> 43 |   expect(paragraph).toBeNull();
  44 | });

Reproduction:

https://codesandbox.io/s/2zwjwyp13n

Problem description:

First off, I'm new to react-testing-library. I read @kentcdodds philosophy - this quote I totally agree with: "The more your tests resemble the way your software is used, the more confidence they can give you.".

That said, I was expecting in the tests above that the given elements would not be able to queried because they are hidden/non-visible/off-screen. I dug around in the dom-testing-library code and noticed that querySelector is still being used but I don't understand how this implementation accounts for elements that the user cannot see, like things that are off-screen, hidden, etc.

I would absolutely love to be able to write tests that exercise a product the way a user would interact with it. But I'm unsure if I can trust dom-testing-library in this sense.

Outside of hidden/off-screen elements cases (and not related to this issue), what about clicking on elements that are unclickable? This is a (surprisingly) frequent z-index bug I've seen. fireEvent wouldn't really cut it. There are lots of edge cases here...

Suggested solution:

I realize that it would be quite difficult to account for all of the cases where an element can't be queried (off the top of my head - opacity = 0, transforms off screen, scaled down, filters, clip-path, etc.) But I would expect at least display = none, visibility = hidden, and off-screen elements to be accounted for. I think this can be handled through getBoundingClientRect but that would sacrifice perf. Maybe this is something that a user could toggle through render?

[sompylasar]

If only Cypress had its DOM query functions reusable... It has functions that check exactly what you're describing. CC @bahmutov

[adiun]

Cypress is really impressive. (Looking at https://docs.cypress.io/guides/core-concepts/interacting-with-elements.html#Actionability). FWIW for handling off-screen I worked around this by using wait with an expectation callback to check if an element is within a container. Would be awesome if I could avoid wait though.

[kentcdodds]

Hi @adiun! I think this would be really cool. I've definitely run into this problem and it's actually why this exists (because my getByText query in cypress-testing-library was returning stuff in a script tag 😱). I don't have the time to work on or investigate it myself, but if someone wants to do that and open a PR that would be very welcome.

Another idea if we can't work this out is to improve toBeVisible from jest-dom to support handling more cases.

[alexkrolick]

In general JSDOM out of the box doesn't support screen sizes very well (I think the default size for all elements is 0x0) nor does it parse CSS stylesheets that can affect visibility of an element (in fact this is probably the primary factor). I think you'd be better off with an in-browser test like Cypress.

[kentcdodds]

I agree 👍

Though I wonder if there are some utilities that we could expose if we're in an environment that does do those things (like the browser). 🤔

[kentcdodds]

Or even better, if there were a way we could filter the elements that are returned by the queries (as originally suggested) if we are able to determine their visibility (because we happen to be in an environment that provides us with that information).

I think this is definitely doable if someone wants to try.

[gnapse]

I wonder what's the definition for "not visible" in this context. For instance, given the first test example above:

test("off-screen element not queryable", () => {
  const { queryByText } = render(
    <div style={{ height: "150px", overflow: "auto" }}>
      <div style={{ height: "100px" }}>Paragraph-1</div>
      <div style={{ height: "100px" }}>Paragraph-2</div>
      <div style={{ height: "100px" }}>Paragraph-3</div>
      <div style={{ height: "100px" }}>Paragraph-4</div>
    </div>
  );
  const paragraph = queryByText("Paragraph-4");
  expect(paragraph).toBeNull();
});

If we change the overflow style to scroll, then would we consider Paragraph-4 to be visible again? Because in that case it can easily become visible.

Also, would a screen reader not read Paragraph-4 above? Does a screen reader has the concept of screen an element sizes into account?

[kentcdodds]

I think that defining "not visible" as something that a screen reader wouldn't read and/or something you cannot find via ⌘ + f.

[adiun]

If we change the overflow style to scroll, then would we consider Paragraph-4 to be visible again? Because in that case it can easily become visible.

I would actually say no in this case, because from the POV of the user, they don't see it. You said "become visible" which implies that it's not visible yet to the user. For more context, I'm specifically testing some custom auto-scroll functionality I wrote so I want the user to click on something that auto-scrolls that div, bringing Paragraph-4 into view and I want to verify that through a test. Maybe that's out of scope for this library though and I'd have to resort to higher-level UI testing frameworks like cypress which is fair...

[gnapse]

Maybe I was not clear enough, because of the different uses we can give to the adjective "visible". Yes, I said "become visible" and in that case I said it in the sense of it being actually drawn in a screen. But the sense of visible that this library aims for is more than that. Under no circumstance we should consider an element that's out of view just because of a viewport size constraint to be "invisible" and not considered in the queries. Even if jsdom allowed for it.

As Kent said (which aligns with what I was aiming for when I questioned the concept of visibility for the queries): if a screen reader reads it out loud, and/or you can find it via Cmd/Ctrl+F, then it's visible.

I just tested the markup of your first example, and both things happen. I'd say that that content is visible.

[adiun]

Thanks for the explanation... that is an interesting requirement. Why is that though? Cypress regards a scrolled-out element as "not visible". When I think of "The more your tests resemble the way your software is used..." I think of a user seeing a panel that doesn't have "Paragraph-4". If they want to click on Paragraph-4, they have to scroll that panel. But I can see how the screen reader/Cmd+F definition can make the implementation more straightforward re. accessing accessible elements. Is that the main reason?