Add Seek for Gain

[OlegOAndreev]

Currently Gain is non-seekable which requires using the SpatialSceneControl::play_buffered instead of SpatialSceneControl::play, which is both less ergonomic and slower (even with optimizations).

OTOH, per-signal gain seems to be standard in many other libraries, e.g. kira, soloud, fyrox-sound.

We can either make Gain seekable or add set_gain (and related methods) to SpatialControl. If we choose to impl Seek for Gain there is a question of whether we should rewind the gain when we seek back in time.

[Ralith]

there is a question of whether we should rewind the gain when we seek back in time.

This is the heart of the problem, and is a design problem that I've wrestled with throughout oddio's development. To generate stereo audio, SpatialScene must sample every Signal twice, over slightly different, but typically overlapping, time intervals depending on the relationship of a given Signal to the listener.

For immutable signals this is easy: we just skip the playback cursor around as needed with Seek. However, if signals are mutable, seeking backwards and resampling the same period may give different results, with consequences ranging from confusing the impression of space (if a sound to your right is louder in your left ear because its gain is being raised dynamically) to severe glitches (if you sampled a Stream which then discarded that data from its ring buffer). Implementing Seek means promising this won't happen, but a filter like Gain cannot provide such a guarantee without storing every change ever applied, which is both difficult and undesirable. By sampling into an intermediate ring buffer that we can then resample freely, we guarantee good behavior for arbitrary signals with only a modest overhead.

A possible solution is replacing Seek with an associated Cursor type that filters like Gain could store state in, allowing SpatialScene to track two cursors for every signal natively. However, there are some drawbacks:

• Significant increase in complexity
• Stream would need to somehow track the position of the earliest of all live cursors to benefit

I have hesitated to pursue this further because the motivation is unclear to me. In oddio, it is trivial to apply dynamic gain after mixing, which is both more efficient and obviates the entire problem. Given this option I'm not sure why you'd want to dynamically adjust the gain of individual sounds. Game UIs tend to, at most, provide a handful of volume sliders, which might correspond to the output of a few discrete intermediate mixers.

Baking additional features into SpatialScene could hack around the problem by allowing state management across multiple samplings to be hard-coded, but is not extensible or composable, and forces cost/complexity on everyone regardless of whether they need it. Alternative ideas to abstract the notion of mutable signal state are welcome.

[OlegOAndreev]

I decided to optimize the play_buffered in #92.

As for the general idea of keeping SpatialScene small and clean, this is obviously your call, I just feel that SpatialScene has a lot of "essential" complexity, such as various types of distance attenuation (at least linear, exponential, inverse), constant power panning, distance-based low/high pass filtering, occlusion volumes/attenuation shapes etc.

Should I close this issue?

[Ralith]

You're right that there's significant complexity intrinsic to spatialization. That's why I'd like to keep complexity that isn't intrinsic to spatialization factored out as much as possible; it's liable to become a big enough monolith as it is.

As outlined above there's a genuine hard design problem here, so we should keep it open unless it ends up mooted somehow.

[Ralith]

Thinking on this a bit more, the associated-cursor approach also isn't quite correct, because cursors at different points in time will observe changes in controls simultaneously, and therefore yield inconsistent results. This still avoids discontinuities, so maybe it's okay, but it's not ideal. Another approach might be to introduce an explicit notion of a state change at a particular time, and defer new state changes until any existing one has been passed by all live cursors. Feels kind of weird to try to abstract over ideas like cursor quantity when there's only one known use case... not sure how I feel about baking in knowledge of spatialization everywhere either, though.

One other bit of food for thought is that reverb will mean buffering everything anyway, and spatial scenes will probably always want reverb, so bending over backwards to avoid buffering is perhaps completely futile in the long run.