Make MIDIAccess transferable and exposed to workers

[mjwilson-google]

This is to fix #99.

Some things to note:

• I wrote in the issue "WorkletGlobalScope" but I think it's supposed to be Worker, correct? I will edit my comment in the issue if so.
• I don't fully understand the implications of these tags or how much implementation work it will be, so please comment if there is anything that should change or if there is anything missing.

[cwilso]

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

[padenot]

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

We've discussed this at TPAC and concluded this is best done by authors, like it is in native code.

[padenot]

I'm wondering if we should make it transfer-only, and refuse regular cloning. I think it would simplify things somewhat.

[hoch]

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

Re: @cwilso - What @padenot stated above means that the group believes this works better with Worker and SAB. Also the feature set in the WorkletGlobalScope is intentionally minimum, so we want to make sure all interfaces used by Web MIDI are properly supported by it.

If incoming MIDI events are bound to AWGS, we'll see a lot of bursty actions in the performance of audio callback. Creating new events is "allocation", and we don't want to do that on the RT audio thread.

I'm wondering if we should make it transfer-only, and refuse regular cloning. I think it would simplify things somewhat.

@padenot Do you have any example of that?

@mjwilson-google Have we confirmed that all the interfaces in the Web MIDI API are supported in the WorkerGlobalScope? I am assuming it does, but it doesn't hurt to check. :)

[cwilso]

@hoch hmm, yeah, you're right, makes sense. I retract my previous statement. :)

[mjwilson-google]

@hoch I don't know how to confirm that the interfaces are supported in WorkerGlobalScope. How do we do that?

[hoch]

From the spec(https://webaudio.github.io/web-midi-api), I see these interfaces:

5.1 MIDIInputMap Interface
5.2 MIDIOutputMap Interface
5.3 MIDIAccess Interface
5.4 MIDIPort Interface
5.4.1 MIDIInput Interface
5.4.2 MIDIOutput Interface
5.5 MIDIMessageEvent Interface
5.6MIDIConnectionEvent Interface

This PR only has a change for MIDIAccess, so it needs to be expanded to add the Worker exposure (e.g. [Exposed=Worker]) to all the interfaces.

[mjwilson-google]

This PR only has a change for MIDIAccess, so it needs to be expanded to add the Worker exposure (e.g. [Exposed=Worker]) to all the interfaces.

Done. Is it fine to keep Transferable only on the MIDIAccess interface?

[padenot]

Done. Is it fine to keep Transferable only on the MIDIAccess interface?

In terms of developer ergonomics, I think it's fine.

The author performs the request from the main thread. This deals with the all the permission business. The author then transfers the MIDIAccess object, and does all the midi things in the Worker. This means that there is generally only one thread that deals with MIDI (it's always possible to get other MIDIAccess however).

We can also decide to allow sending ports and such to the worker, but I think I like that style less.

[hoch]

LGTM

[mjwilson-google]

@padenot How does the current state look? Do you want to explore making the object transfer-only? Would the way to do that be to mark it as non-serializable, to disable cloning?

We will need a follow-up PR to further define the behavior when transferred before closing the linked issue.

[svgeesus]

Should this be merged? We have an approval from @hoch and a "looks okay" from @padenot

[mjwilson-google]

Yes it should be merged. I have been waiting to update the working draft first. Sorry about the long delay.

[svgeesus]

Will merging trigger the automatic re-publishing, or is that not set up yet?

I really want to see a WD that is more recent than 2015 :)

[mjwilson-google]

Will merging trigger the automatic re-publishing, or is that not set up yet?

I really want to see a WD that is more recent than 2015 :)

I just put up pull request #262 to add the auto-publish workflow. I think I need your help for the Echinda token; could you please take a look?

Once that is merged, then yes merging this should automatically update the Working Draft and we can start merging other changes more quickly again (which will also update the Working Draft).

Thank you for checking in on this, and sorry again for the long delay.

[svgeesus]

So I am seeing the new ED
https://webaudio.github.io/web-midi-api/
W3C Editor's Draft 04 July 2024

But https://github.com/WebAudio/web-midi-api/actions/runs/9794115046/job/27043463563 says TR publication was "skipped". And nothing at
https://lists.w3.org/Archives/Public/public-tr-notifications/2024Jul/

I don't understand why.

[mjwilson-google]

I think I need to change ECHIDNA_TOKEN in auto-publish.yml to WEB_MIDI_ECHIDNA as per #262 (comment) -- I will raise a separate PR for that.

[mjwilson-google]

After updating the token name the new WD is live. Thanks for the support @svgeesus!

[kyledecot]

@mjwilson-google now that this has been published are there plans for Google Chrome to support this feature? I attempted to transfer a MidiAccess object to a worker but got:

Uncaught DataCloneError: Failed to execute 'postMessage' on 'Worker': Value at index 0 does not have a transferable type.

I opened an issue for this at https://issues.chromium.org/issues/379805727 so mostly just posting this here for visibility but would love to know more if you have any information.

Great work on this everyone! 🎉 This will definitely open up some great possibilities once browsers support this.

[mjwilson-google]

@kyledecot Thanks for filing the Chromium issue!

I don't have a timeline for implementing this in Chromium yet. Right now I'm using the "Web MIDI time" I have to work on the spec, and plan to focus on implementation gaps once the spec is in CR review.

[sebs]

@mjwilson-google cool that you do so. we just rolled a 99 need on this item. ;)