Add support for testing the WebExtensions API in WPT #219
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,56 @@ | ||||||
| # RFC #219: Add support for WebExtensions in WPT | ||||||
|
|
||||||
| ## Summary | ||||||
|
|
||||||
| The WebExtensions API extends the capabilities of the browser. Adding support | ||||||
| for WebExtensions in the web platform tests will increase interoperability | ||||||
| and will help drive the standardization of this API. | ||||||
|
|
||||||
| This RFC proposes adding a new `testharness.js` test type, `.extension.js`, to handle | ||||||
| testing this API, in addition to using `testdriver.js` to load and unload extensions. | ||||||
|
|
||||||
| ## Details | ||||||
|
|
||||||
| Using `testdriver.js`, we added support for testing the WebExtensions API by loading | ||||||
| a web extension designed to test the functionality of a specific API. | ||||||
| The extension will be loaded after the tests begins, and unloaded before the | ||||||
| test is finished. | ||||||
|
Comment on lines
+16
to
+17
There was a problem hiding this comment. We probably don't need this; technically people writing tests can do whatever, and I don't think we need to justify this. I'd probably add:
|
||||||
|
|
||||||
| Most of the test execution is handled within the extension, via the | ||||||
|
There was a problem hiding this comment. I think we need a sentence preceding this, maybe the one from the following paragraph?
|
||||||
| [browser.test](https://chromium.googlesource.com/chromium/src.git/+/master/extensions/docs/testing_api.md) | ||||||
| API. We’ve elected to use these APIs since all participating browser vendors use | ||||||
| `browser.test` internally and they can easily port over existing tests to | ||||||
| the web platform tests. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| Because these tests won’t leverage `testharness.js` directly, we’ve introduced a new | ||||||
| `testharness.js`, `.extension.js`, that will create the necessary boilerplate to | ||||||
| convert the `browser.test` assertions into the corresponding assertions in the test | ||||||
| harness. | ||||||
|
Comment on lines
+25
to
+28
There was a problem hiding this comment. We should probably care to some extent about how easy/difficult it is to write tests without using Rather than putting all the code necessary to map from |
||||||
|
|
||||||
| A proposed patch is available at https://github.com/web-platform-tests/wpt/pull/50648. | ||||||
|
|
||||||
| ## Alternatives considered | ||||||
|
|
||||||
| We considered loading the extension statically before each `testharness.js` test is run, | ||||||
| but we decided against that since `testdriver.js` will allow the tests to drive the | ||||||
| browser, and in the future, test functionality such as opening popups and clicking | ||||||
| menu items. | ||||||
|
|
||||||
| We considered adding a new test type rather than using JavaScript tests, but we decided | ||||||
| against it because it was a lot more work compared to using `testharness.js`. | ||||||
|
|
||||||
kiaraarose marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| ## Risks | ||||||
|
|
||||||
| There are two potential concerns with this implementation: | ||||||
|
|
||||||
| 1. We have no precedent for tests run via a Classic command in some user agents | ||||||
| and BiDi command in others. However, the Classic implementation of loading | ||||||
| and unloading extension is modeled on the BiDi implementation, so we expect the | ||||||
| behavior to be the same. The Classic implementation is defined in the WebExtensions Community Group | ||||||
| [here](https://github.com/w3c/webextensions/blob/main/specification/webdriver-classic.bs), | ||||||
| and the BiDi implementation is defined | ||||||
| [here](https://www.w3.org/TR/webdriver-bidi/#module-webExtension). | ||||||
|
|
||||||
| 2. Another concern could be with using `browser.test` assertions and mapping them to | ||||||
| `testharness.js` assertions. With `testharness.js` not in charge of generating assertions, | ||||||
| we might end up with less useful failure messages. | ||||||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This technically isn't a new test type (because the manifest will still make it a testharness test), it's just server-side support for creating them from a pure JS file.
We could potentially split this out into a separate RFC, but that's probably only worthwhile if we start having significant debates about whether/what that support should be.