MSFT_audio_emitter extension

[najadojo]

We at Microsoft have been developing this extension for use in various products and scenarios. We are sharing to document software we will be shipping and to get feedback on the extension design. Its possible that this extension maybe useful for other vendors as evidenced by #753.

[lexaknyazev]

May I nitpick about audio data formats?

• Why it's only audio/wav?
• According to IANA registry, audio/wav is not a valid registered MIME type (despite being widely used). Other seen options are audio/vnd.wave and audio/x-wav.
• Assuming that WAV means RIFF container, which of many possible WAV encodings are supposed to be supported?

[najadojo]

We picked wav because its commonly available and trivial to decode almost anywhere. Continuing with the runtime friendliness of glTF and the desire to use linear PCM decoded audio data at runtime it made reasonable sense. Our least capable client is able to decode wav with WAVE_FORMAT_PCM, and WAVE_FORMAT_IEEE_FLOAT. I'll update the mime type value.

[najadojo]

I created a demo video that shows how this extension can be used in Babylon.js.

[pjcozzi]

@amwatson and @tparisi had an interest in spatialized audio a couple of years ago, #753. Perhaps they want to chime in here?

[pjcozzi]

@najadojo is this spec/schema still up-to-date with the Babylon implementation?

@sbtron @lexaknyazev OK to merge from your perspective? Perhaps the samples belong in the glTF-Sample-Models repo?

[najadojo]

Yes Babylon.js and Windows Mixed Reality are using this extension as its presented here.

[pjcozzi]

@najadojo very nice!

@sbtron @lexaknyazev please merge when you are ready.

[merlosy]

I'm interested in this feature :) plz

I don't know if you knew: Babylonjs has already implemented it: https://viewer.babylonjs.com/ufoexample

[faridskyman]

Video demo was great, having a 3d object with animation that also plays sounds is a terrific idea. However this thread is dead, Khronos people? what happen? u guys dont like this? then do u have a better idea? or no action?
can we have this or u guys comeout with a standard?
else its strange, we get user to upload audio and we try to sync with with the animation?
dont seem a good idea..
hope khoronos will reconsider.. its 2022, 4 years has passed...

[bghgary]

See #2137.

[aaronfranke]

This is inconsistent with KHR_lights_punctual which defines spotlights as facing -Z. Is it expected that conical audio should be pointing the opposite direction as conical lights?