Support querying by ARIA role instead of relying only on HTML semantics?

[stanlee974]

Hi !
First off, big fan of Interactors – I really like how it brings test interactions closer to the actual user experience. It’s a clean and thoughtful approach. I had a quick idea I wanted to run by you:

Would it make sense to support selecting elements by their ARIA roles (native role or explicit role e.g. role="button") in addition to using native HTML semantics like <button> or <nav>?

Many modern apps, often run into custom components or elements that rely on ARIA roles for accessibility rather than native HTML tags. And since ARIA roles are central in both WCAG and W3C guidelines, supporting them would make Interactors even more aligned with accessibility best practices.

For example:

• a <div role="button"> that behaves like a button
• a custom input field using role="textbox"
• menus or tab lists that follow ARIA patterns, not HTML5

I'm also seeing a strong alignment here with the testing philosophy behind UUV (User-centric Usecases Validator), which is built around role-based querying and human-readable tests that reflect what real users experience. Two tools, one vision, just not the same battlefield.

What do you think?
Would this be something worth exploring or supporting natively?

Thanks again for the great work you're doing here – this project really pushes testing in a meaningful direction.

[cowboyd]

@stanlee974 I think this is a great idea. you could testit out by extending the button selector:

import (Button as $Button } from "@interactors/html";

export const Button = $Button.extend('AriaButton')
  .selector('button,input[type=button],input[type=submit],input[type=reset],input[type=image],[role=button]')

[cowboyd]

If this strategy works out, would love to integrate it into the core

[stanlee974]

Thanks for the suggestion, but unfortunately, it doesn't work as expected. It recognized the button but, not the accessible name and disable information.

I believe several things need to be reconsidered if we want to properly support ARIA roles.

---

Just adding [role=button] to the selector is not enough.
That <div role="button"> example won't behave like a real Button Interactor, because under the hood, isButtonElement relies only on tagName.

Also, using innerText to resolve the label isn't always reliable anymore.

---

As you can see, there's a specific priority order when computing an accessible name for elements:

1. aria-labelledby
2. aria-label
3. Inner text content
4. title attribute

This order differs depending on the role, and follows a much more complex algorithm defined here:
W3C computation accessible name

The example I shared (from a screenshot) is the simplest case, the text is directly in the element. But it gets more complex when labels are referenced from other nodes or dynamically rendered.

---

In our project, we used Testing Library to avoid implementing the full accessible name computation ourselves.
However, even Testing Library delegates this to another library (dom-accessibility-api) that has known limitations for some roles.

---

The fix you proposed is definitely a first good step, but proper role support will likely need:

• A rework of how Interactors resolve elements (beyond tag names)
• A way to compute the accessible name more robustly (maybe relying on the same engine as Testing Library?)
• Careful handling of edge cases depending on roles

In short: supporting ARIA roles isn't a simple change, but it's a valuable direction, especially if we want Interactors to reflect how users actually experience interfaces.

[taras]

Hi @stanlee974,

Welcome and it makes us happy that you like interactors.

Like Charles said, it makes a lot of sense to use ARIA roles this way.

How much do you think we should bring accessibility elements into interactors in your opinion?

[taras]

I'm glad that our vision resenates with you.

When you talk about "decoupling declaration from implementation", do you envision non-technical people (e.g. product, QA, accessibility specialists) being able to author tests in that syntax, or is it still primarily developer-facing?

Ultimately, the goal is to create a shared artifact that is erogonomic for everyone involved in the process. There are hover some practical limitations today that require us to be strategic about our approach. The reality is that developer's perception of e2e testing and the detrimental impact of developers treating e2e tests as someone else's problem is a much harder nut to crack than the needs of other personas. The impact of slow feedback caused by not having tests or having bad tests is percieved by the organizations as a much bigger problem than tester's and product's needs. We need to first win the hearts and minds of developers and everything else will follow.

How do you see accessibility fitting into that model? For example, if tests are declarative and evaluate to data, could we embed accessibility requirements (roles, ARIA patterns, WCAG checks) directly into that data model so they become a first-class citizen?

Agree. We believe accessibility should be built into the system. One way that you can see that accessibility is built into the system is when it's easier to test accessible applications than non accessible applications. The reality is that most applications are not going to be accessible on day one. The system needs to allow testing inaccessible applications, but the tests have to make it very clear when relying on an inaccessible behavior.

One of the reasons I'm asking is because with UUV, we've been trying to formalize user intent in a way that's both readable for product teams and grounded in accessibility semantics (roles, accessible names, APG patterns).

Agree. I was looking at UUV's Step Definitions and they look very similar to the test output produced by test execution of a proprietery version of BigTest that we built for one of our clients. In that system, test authors do not write words, they write declarative code that is used to generate test output. For UI tests, most of the actual text is generated from UI Interactors output. This makes test output very consistent and eliminates the need for policing how test authors write tests. We allow TypeScript compiler to ensure that the test makes sense and test output is derived from test data.

Your approach sounds like it could provide a much stronger foundation for that kind of user-focused testing.

We could build ideas from UUV right into the system used to generate test output.

Would love to hear more about the "few more foundational ideas" you mentioned, especially if any of them could help bridge the gap between confidence for product and ergonomics for developers, without leaving accessibility as an afterthought.

I mentioned earlier that tests should make it very clear when it's relying on inaccesible behavior of the application. For accessibility problems to be addressed, they first have to be visible. Test authors, developers, product people and leadership usually look at different artifacts. Test authors look at tests, developers look at the application code, product people look at UI and acceptance criteria, leadership look at charts. To improve accessibility, we need a way to connect these personas. The one thing they all have in common is the ticketing system. We can use the ticketing system (and we have successfully in the past) to our advantage.

The process looks something like this,

1. The testing tool must be designed assuming accessibility.
2. When a test author can not write a test because the UI is inaccessible, this is an accessibility bug.
3. Finding and fixing bugs is usually done by different teams or people.
4. The test author needs a way to "workaround" the accessibility bug so they can proceed with their work while the bug is addressed. We can elevate the concept of a workaround to the test syntax.
5. A workaround is a custom test syntax.
6. For a test author to use the workaround test syntax, they must annotate the test step to make it explicit that they're relying on a workaround and provide a URL of a ticket that will track the workaround.
7. Used workarounds are included in test results and they can be tracked on a dashboard.
8. When the accessibility bug is fixed, the workaround usage diminishes.
9. When workaround usage is at 0, it can be removed from the code.

We used workarounds to deal with all kinds of bugs in applications, but it emerged out of a need to solve a real business problem. The testing syntax and test runner must make it trivial to create new syntaxes to solve real business problems.

When we finished writing the last version of BigTest, we realized that we had good ideas but we didn't have the full picture because we were too focused on UI testing. We created a Playwright and Cypress competitor instead of a tool that solves business problems. 3 years later, I believe we now know what it'll take to create a platform that is both powerful and flexible enough to change the status quo.

[stanlee974]

That's a fascinating deep dive thanks for taking the time to lay it out so clearly.

I really like the idea of a shared artifact that's ergonomic for all personas, but with a deliberate first step of "winning the hearts and minds" of developers. That's a reality I've seen as well: if developers see e2e as painful or "not their job", the rest falls apart.

Your point on accessibility resonates even more with me now. Making it easier to test accessible applications than inaccessible ones and explicitly marking workarounds feels like a game changer. In UUV, we've been experimenting with similar principles:

• role-based selection
• APG pattern awareness
• explicit warnings when the DOM doesn't expose an accessible name

The idea of elevating "workarounds" to a first-class concept in the syntax is very interesting. That's actually the kind of visibility product, QA, and leadership could all understand especially if it ties into the ticketing system.

I'd be curious to know:

1. How do you envision surfacing these workarounds in a way that's visible and actionable for non-technical stakeholders?
2. In your model, could accessibility metadata (roles, APG compliance, WCAG checks) become part of the test output "data" so it flows naturally into reports, dashboards, and tickets?
3. For the new BigTest, are you aiming for this platform to be extensible enough for domain-specific layers like UUV to plug into it?

On our side, we've seen role-based validations uncover issues that slipped through purely functional tests. I'd be happy to share some real cases they might spark ideas as you refine the foundation.

[taras]

How do you envision surfacing these workarounds in a way that's visible and actionable for non-technical stakeholders?

Use of workarounds should be tracked as individual metrics on a dashboard. The number of actively used workaround gives the organization a heartbeat for quality of their product. The dashboard are a trailing indicator - when the source of the problem is fixed which eliminates the workaround from being necessary, the metric goes down to a 0.

In your model, could accessibility metadata (roles, APG compliance, WCAG checks) become part of the test output "data" so it flows naturally into reports, dashboards, and tickets?

Extending the test syntax is relatively straightforward to add metadata because the syntax evaluates to data. The backend part of this is some of the secret sauce that I'm happy to talk about in person, but I probably won't write about it until after the alpha release of BigTest.

For the new BigTest, are you aiming for this platform to be extensible enough for domain-specific layers like UUV to plug into it?

Yes. The caveat being we're going to start with BigTest as a toolset for building test harnesses. It'll require a lot of customization. Not more than it takes to build a good testing harness today, but it'll be development of higher engineering sophistication. Right now, companies think of e2e testing as the process where "the most technical tester sets up Playwright and a bunch of people who are not developers start writing a bunch of tests." Predictably, it results in e2e tests being unreliable, slow and difficult to work with.

We've seen repeatedly that when a senior engineer or a principal engineer builds a testing harness the result can be good, but it's difficult to scale because a home grown testing harness never gets enough engineering attention. What's missing is a toolset makes it difficult for people to make mistakes - that's also known as creating a pit of success. We're going to focus on building this toolset and educate people on how to use it. In the process, we can build the extensibility model to support UUV style plugins.

[stanlee974]

That makes a lot of sense, particularly the point about harnesses built by senior engineers being excellent but difficult to replicate. A "pit of success" approach sounds exactly like what's needed today.

Regarding the workaround metrics: I like the idea of using them as an indicator of quality. That's the kind of visibility that leadership usually pays attention to. Do you see those metrics as purely quantitative, e.g. the number of active workarounds, or would there also be a way to categorise them, e.g. accessibility vs. performance vs. functional gaps? This categorisation could be really powerful for identifying systemic issues.

The extensibility angle is exciting, too. From the UUV side, I can see a natural fit if the model allows plug-ins to enrich test data with accessibility semantics.

Either way, I'll be following BigTest's progress closely, so please don't hesitate to reach out if you'd like to discuss UUV ideas.

[taras]

Do you see those metrics as purely quantitative, e.g. the number of active workarounds, or would there also be a way to categorise them, e.g. accessibility vs. performance vs. functional gaps?

This makes a lot of sense. We've only used functional workarounds so far, but it seems probable that categories will emerge as we apply this idea to more use cases.

This categorisation could be really powerful for identifying systemic issues.

I agree.

The extensibility angle is exciting, too. From the UUV side, I can see a natural fit if the model allows plug-ins to enrich test data with accessibility semantics.

Sounds good.

Either way, I'll be following BigTest's progress closely, so please don't hesitate to reach out if you'd like to discuss UUV ideas.

The first step will be to introduce the new test syntax and tooling around it. I'm planning to start working on it in October after a vacation :)