[WinMM] Testing Main: Sysex messages are split in chunks of 6 bytes

[jmelas]

MIDI/src/api/Client/WinMM/MidiSrvPort.cpp

Line 688 in abc0bb3

RETURN_IF_FAILED(CompleteLongBuffer(MIM_LONGDATA, position));

I am testing current implementation from main (latest commit: abc0bb3)
This line forces incomplete sysex messages to be sent. This is different behaviour from old WinMM implementation.
This behaviour was introduced here: b60c2f6

I've tested removing this line and now I can receive sysex messages exactly as before!

[Psychlist1972]

Hi JM

How are you testing that current commit? It's not in Insider builds yet and that code is in a file that is protected in System32 and can't be replaced without special tools.

In any case, please share your repro steps or code with me so I can look at it. Thanks!

[jmelas]

Hi Pete!

Thank you for your answer!

I built the code from the repo (Midi2.sln).
The only change I had to make to build was to update .NET version in global.json from 8.0.300 to 8.0.406.
I am deploying by running the scripts in build folder replace_running_service_x64.bat and replace_wdmaud2_x64.bat.
For wdmaud2.drv deployment to succeed I had to take ownership of C:\Windows\System32\wdmaud2.drv and change its permissions. Then I had to copy xcopy.exe to the build folder as sfpcopy.exe. It does the job! :-)

To receive sysex input from old WinMM I am using MIDIHDRs with dwBufferLength 4096.
Since my buffers are that big, every MIM_LONGDATA message in the callback delivers a complete sysex message starting with F0 and ending with F7. Of course if I used a smaller dwBufferLength I would receive the messages split in chunks.

So the old WinMM implementation never sent incomplete sysex messages (unless of course a sysex was bigger than its internal buffer, I see in the new code that this size is 4096 and I think it's important that this is the same with old implementation).
So I think new WinMM code should do the same: as long as a sysex size is less than 4096, it should be delivered in one single MIM_LONGDATA call. If this behaviour changes, it could break lots of apps if they are relying on the fact that MIM_LONGDATA always delivers a single complete sysex message.

To test sysex I think you need a synthesizer. I have several Yamaha synths as I am creating editors for them: https://www.jmelas.gr
You could try sending an ID Request message F0 7E 00 06 01 F7 and check what is received in MIM_LONGDATA.
The answer of ie Motif XF6 is F0 7E 7F 06 02 43 00 41 41 06 05 00 00 7F F7.
Current main will send this as three MIM_LONGDATA calls:
F0 7E 7F 06 02 43 00
41 41 06 05 00 00
7F F7
instead of one complete call (old WinMM behaviour)
F0 7E 7F 06 02 43 00 41 41 06 05 00 00 7F F7.

I hope I explained well, if you need anything else please tell me!
Thank you very much!
Kind regards
John Melas

[Psychlist1972]

Thanks John. Nice to know xcopy works in this situation. I thought for sure it didn't. :)

I need to look back through this logic and remember why I put that in there, if it was just to deal with a long-standing bug (more on that in a second), I'll pull it.

The big difference right now is that the service works with UMP data format internally, no matter what the source or destination of the data is. The UMP data format has 6 SysEx bytes per 64-bit packet. Those are received one at a time in the callback code, which is why you're seeing 6 bytes at a time.

One of the long-standing WinMM bugs was that a SysEx message from a device would never make it through if there was no F7. I'm trying to remember which devices do that, but it was more common than you'd expect. Those would work on macOS but not on Windows. But a callback every 6 bytes isn't optimal either. The right approach is likely some sort of watchdog timer, but we won't have anything like that, at least in the first release.

Between this and naming, seems every thing we try to fix in WinMM ends up breaking backwards compatibility in some way. More reasons to try to get folks using the new SDK instead of the old WinMM API. :P

[jmelas]

Thank you Pete for the explanation!

I think it's more important to keep old behaviour intact than adding support for misbehaving devices.
Not only a callback every 6 bytes is suboptimal but it also violates the WinMM API design which works with user allocated buffers. If it wanted to use only 6-8 bytes per call then the MIDIHDR struct would not point to user allocated memory, it would simply use 64 bits to store the data.

Great idea for a watchdog timer to fix misbehaving devices!

I also agree that naming should be exactly the same as before!

[Psychlist1972]

I've removed the buffer completion code. It won't be in the next Canary release because I made the change this week.

So far, I haven't seen any unexpected issues.

[jmelas]

That's great, thank you Pete!

[Psychlist1972]

Correction, We're another week or two from this being in Canary. Sorry for getting that wrong.

[jmelas]

That's even better, thank you Pete!