Description
In our packages we accept axeSource
as a way to pass in the version of axe to run. However we don't validate that the string passed in is either 1) valid JavaScript or 2) is an axe-core script. This can lead to a problem where an invalid string passed into the builder can cause tests to break in ways that you wouldn't expect and wouldn't be able to debug easily.
For example, using the following code:
const AxeBuilder = require('@axe-core/playwright').default;
const playwright = require('playwright');
describe('tests', () => {
let browser, page, axeBuilder;
this.timeout = 10000;
before(async () => {
browser = await playwright.chromium.launch();
const context = await browser.newContext();
page = await context.newPage();
axeBuilder = await new AxeBuilder({ page, axeSource: "global.axe = 'foo'" });
});
after(async () => {
await browser.close();
});
it('should run', async () => {
await page.goto('http://localhost:9876/test/playground.html');
try {
const results = await axeBuilder.withRules('heading-order').analyze();
} catch (e) {
throw e;
}
});
});
Produces an error in the test about an axe function not existing.
However, since the cause of the problem is in the before
but the manifestation of the problem is in the test, the error message does not help you understand why it's failing.
Instead we should do some simple validation of the axeSource
string provided so the user gets immediate feedback of when there's a problem. A simple example of that could be (we'll probably want to check more than just version
though):
try {
const axe = eval(this.source);
assert(axe.version, 'Axe Source provided is invalid');
}
catch (err) {
throw new Error('Axe Source provided is not valid JavaScript', err.message);
}
NOTE: we should not use eval
in the code as that's a security problem. Just wanted to use it as an example of one way to do script validation. We could do something similar by opening a blank page and evaling the script there and checking for version
.