RFC 90: [testdriver] Primitives for cross-context messaging

[jgraham]

Add send()/poll()/recv() functions that can be used to pass arbitary messages to
other testdriver contexts, to help with test scenarios requiring multiple origins

[ArthurSonzogni]

nit: "receive"

(×2)

[ArthurSonzogni]

Shouldn't those specify the uuid of the queue your are receiving messages?

It is highly important to be able to listen to different queues. You might want to control multiple context or run multiple subtest in parallel. Having multiple queues is very important.

[jgraham]

The way it's set up at the moment, you can only listen for messages sent to your queue. But each message has a src property that tells you where it came from. So you don't supply a UUID here because it's implied that it's the one for the current context.

[ArthurSonzogni]

How can this be used to run several subtest in parallel?

Having a single queue per context is a strong limitation. As is, I don't believe this solution could replace code from:

• https://github.com/web-platform-tests/wpt/blob/master/html/cross-origin-embedder-policy/credentialless/resources/dispatcher.py
• https://github.com/web-platform-tests/wpt/blob/master/html/cross-origin-opener-policy/resources/dispatcher.py

[jgraham]

Presumably you could have some code like:

let awatingContexts = new Map();
fn messageFromContext(uuid) {
   new Promise(resolve => awaitingContexts.set(uuid, resolve));
};
async function dispatchMessages() {
  while (true) {
    let msg = await test_driver.recv();
    let waiter = waitingContexts.get(msg.src);
    if (waiter) {
        waiter(msg.data)
    } 
  }
};
dispatchMessages();

And then a test would do something like

// Assume ctx is a uuid
await test_driver.send(msg , ctx);
let response = await messageFromContext(ctx);

So I don't think it's impossible, but I agree there's some additional overhead from the fact that the message queues are per context. But it feels like the per-context model is closer to the model for other testdriver commands. Do you think that adding some additional built-in API surface for this kind of message dispatch would be enough to make the solution work for you, or do you have another suggested approach?

[ArthurSonzogni]

Why does it has to belong to testdriver? I am not sure to understand the benefits. Maybe if I knew, they could potentially offset the drawback mentioned above?

I liked a lot hiroshige@ patches. Keep the message queue API very simple. Every context can pop/push to any arbitrary queue. Messages are simple string.
Then, once you have the capability to communicate, you can build second layers object like the Executor/ExecutorTarget, providing remote code execution, and returning result or errors.

1. https://chromium-review.googlesource.com/c/chromium/src/+/3033199
2. https://chromium-review.googlesource.com/c/chromium/src/+/3055392/2

[ArthurSonzogni]

I forgot... Most COEP:credentials are built this way:

1. Setup: Create multiple context with different policies.
2. Subtest in parallel: execute/test code in the contexts from (1).

So there are multiple message queues to communicate in each pair (driver, helper) contexts.

With the current solution, it is limited to one queue per receiver context. As you said, we can simulate more by dispatching per sender context. So at most we have one queue per (receiver, sender) pair, but this is not enough.

[jgraham]

It doesn't have to of course. But the reason for proposing this is to end up with a unified way of working with remote contexts, rather than having the concept of a remote context with a bunch of functionality like invoking user gestures, or running script, and a semi-decoupled model of messaging where the message queue ids are kind of coupled to context ids (because we give contexts an id through the URL), but anyone with the id has full read/write access. The model here seems closer to other things like postMessage where anyone (more or less) can write but only the target can read.

I suppose another solution to the multi-test problem above would be introducing the ability to create non-context-bound message queues like the MessagePort DOM API. Then a test would create a context, and send a message to it containing the channel id to reply on. The test could await a message on that specific channel rather than doing dispatch on the src on the context-global channel.

[ArthurSonzogni]

I am thinking, what about:

• Make the UUID of "receive" optional. By default it will listen to the current context predefined UUID.
• For other use case, like all mine, we can specify an arbitrary UUID.

[ArthurSonzogni]

nit: "message"

[ArthurSonzogni]

Optional: I like specifying the target context/uuid before the messages.:

test_driver.send(context, `
  Hello from a different context
`);

[jgraham]

All the other testdriver commands take the context as the final argument, which is why I went with that order, even though I agree that this is more natural.

[ArthurSonzogni]

ACK. That's unfortunate, but not a big deal.

[Ms2ger]

Promises? Or challenges?

[jgraham]

It promises some challenges :p (but yes, good catch).

[ArthurSonzogni]

Where does the UUID come from?
You said we can't generate them at will, but there will be instead one per context predefined.
Do we have an UUID for every type of context? (document, sharedworker, dedicatedworker, serviceworker).
How do you convey the uuid from one context to the other?

[ArthurSonzogni]

I guess they come from the uuid params of the URL?

[jgraham]

See #88

[jgraham]

So, to be clear I'm not wedded to the proposed setup. But I'm also skeptical of making the API surface a multi-producer, multi-consumer queue in which we expose a way for anyone with the token to both read and write. The problem with that design is that I fear it will lead to hard-to-debug flakiness where for timing reasons one context reads something that was supposed to be read by a different context, and the whole implicit protocol breaks down. Constraining who can read from a queue, even if multiple people can write to it, seems to encourage more robust designs.

That said, I agree that the proposal doesn't address the use case of multiple tests running concurrently, and all wanting to communicate back to the test context without needing complex dispatch. One way to extend the proposal to support that might be to allow something like [receiver, sender] = test_driver.channel(); remote_context.send(sender); await reciever.recv() i.e. you'd get seperate objects implementing the send() and poll()/recv() parts of the protocol, and it would be possible to send() the sender (but not the receiver) to a remote context so you'd get a unique channel per test to communicate back results. For example, in the test window:

let uuid = "{{uuid}}"';
window.open(`child.html?uuid=${uuid}`, "_blank", "noopener");
let remote = new test_driver.RemoteContext(uuid); // This only implements send()
let [receiver, sender] = new test_driver.Channel()
await remote.send(sender);
// Listen for a message from the remote end
await reciever.recv()

and in child.html:

let test_channel = test_driver.recv();
try {
  let result = await do_test();
  test_channel.send({status: OK, result: result});
} catch(e) {
  test_channel.send({status: "ERROR", message: e.toString()});
}

So test_driver would automatically implement the receiver API for the current context (and we could also make that available in workers / remote contexts), and given a uuid, it would be possible to construct a sender, but not a receiver.

One thing that might be useful here is to look at use cases. For example:

• Test wants to get some data from a browsing context in a different browsing context group in order to make asserts about it
• Test wants to get some data from a worker running in a different origin to make some asserts about it
• Multiple subtests on the same page want to create browsing contexts/workers and get results back from those workers
• Test wants to observe events happening in a remote context/worker (e.g. check that a load event happens)

And maybe some higher level ones:

• Test wants to send over some script to run in a different context/worker and get the result of the execution back
• Want to run assert() functions outside the test window (in a different browsing context group) and communicate back the asserts run and their results (c.f. fetch_tests_from_window etc.).

Does that sound like a reasonable start? What am I missing?

[ArthurSonzogni]

Does that sound like a reasonable start? What am I missing?

I think we both understand each other.

The additional test_driver.Channel() can help, but this adds more work for you and for developers to use. Allowing a uuid params (even optional) would be simpler IMO.
I think you should implement all of your proposals and try them it as a replacement for the two send() receive() functions and executor.html file from:
https://github.com/web-platform-tests/wpt/blob/master/html/cross-origin-embedder-policy/credentialless/resources/dispatcher.py
This might help you figure out what is best, and helps me to realize this is a good fit.

Alternatively, you can also make COOP/COEP tests not to use this proposal, that's also a possibility. This proposal is good on its own, even if we don't manage to use it for the existing tests.

one context reads something that was supposed to be read by a different context.

I would argue that with multiple queues, this precisely avoid having multiple readers reading the same queue.

[jgraham]

OK, so I'm approximately ready to replace this PR with one that proposes a channel-based model.

I've updated the branch at web-platform-tests/wpt#29803 with a WebSockets-backed channel-based proposal and converted a bfcache test and a COOP test to the new model.

The COOP test in particular is very similar in the new model, but I think there are a few quality of life improvements:

• Because it supports serialzing complex data types rather than just strings, we can send functions with parameters to be executed on the remote rather than having to manually construct strings to eval. This means that syntax highlighting &c. works better so it's easier to write tests.
• The use of websockets backed by queues means we don't have to contually poll the server, which ought to reduce resource usage.
• The model is really close to WebDriver-BiDi, so hopefully at some point in the future we can replace the WebSockets backend with a WebDriver backend with minimal changes.

The bfcache test required some additional work, mostly because haing ongoing requests / open sockets prevents pages being added to the bfcache. This required a mechanism to stop new messages being read on the channels before navigation (so that the client websocket doesn't buffer a message, only for it to be lost after navigation). Obviously this is easier with a polling approach since the user code is in control of when we poll for new messages. If it turns out that this finegrained control is necessary, we could use the same server-side message storage approach but add a client API in which the client has to explictly poll() for a new message, causing a HTTP request which reads a message off the queue, if any is present. That could reuse most of the same frontend code.

If people think this is a good option to pursue I'll write up a full RFC for the new proposal, but some more technical details to help evaluate what's going on:

• Messages are stored on queues which are constrained to be multiple producer, single consumer.
• Queues are identified by a UUID.
• On top of the queues we build a channel abstraction. Channels can either send messages or receive messages, but not both. Only only message reader channel may be created per queue.
• Channels can send complex js objects, using a serialization format based on WebDriver-BiDi. It's also possible to send a SendChannel, giving the owner of the destination read channel the ability to send messages to the SendChannel's corresponding RecvChannel.
• Layered on top of channels is a higher-level "testdriver" API (this doesn't actually depend on any other parts of testdriver, so might be badly named). This is accessed via a TestDriverRemote object with executeScript(fn, ...args), executeScriptNoResult(fn, ...args) and postMessage(msg) methods. These are designed for performing actions in a remote context.
• A page with a uuid parameter in the URL can call ctx_channel to create a TestDriverCommandRecvChannel for that uuid. This handles recieving messages sent over the "testdriver" API and actually executing the actions (e.g. running the script and returning the result).
• TestDriverRemote also has a pause() action which causes the reader channel to disconnect until explictly reconnected. This is useful for "send a command but don't allow the remote to process any more commands until after it completes some async action (e.g. navigation) and reconnects".
• There's a closeAllChannelSockets() method which is a low-level tool to ensure no websocket connections remain open. It's kind of a hack for bfcache tests and could probably be improved.

Anyway the general pattern is that in the main test file you have something like

<title>Demo of executeScript</title>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/resources/channel.sub.js"></script>
<script>
promise_test(async t => {
    let page = new TestDriverRemoteContext();
    window.open(`child.html?uuid=${page.uuid}`, "_blank", "noopener");
    assert_equals(await page.executeScript(() => document.body.textContent), "PASS");
})
</script>

And in child.html you have something like

<!-- This is currently needed for reading the uuid, but could maybe be removed -->
<script src="/resources/testdriver.js"></script>
<script src="/resources/testdriver-vendor.js"></script>
<script src="/resources/channel.sub.js"></script>
<p>PASS</p>
<script>
// Listen for messages
ctx_channel();
</script>

[ArthurSonzogni]

I took a quick look. I like the API a lot and how it looks on the existing COOP test above.
Thanks for working on this! This will help writing many tests about navigations / iframes / popups.

I would be curious to see how SendChannel would look like if implemented.

[jgraham]

I'm withdrawing this in favour of #98 which has an alternate approach to the same problem, not based on polling.