Cloud-based streaming comms

[rawnsley]

Has the team considered replacing the mediasoup SFU server with a cloud-based service like Chime?

Advantages could include scalability, cost, server-side mixing, and simplified Hubs Cloud configuration (on AWS at least).

Vendor lock-in could be mitigated with an adapter-based abstraction layer.

[netpro2k]

Chime does look interesting, and definitely seems worth looking into. From a quick poke around it does seem like the basic features needed for Hubs exist, though you would still need a server in the middle for actually making the API calls to amazon to kick off the "meetings" and add "attendees". This could technically be added into reticulum itself or handled in an external service.

A quick poke around at pricing seems like you would need the "pro" tier, which is $3/user/day. You might be able to get away with pooling these "users" so that your daily cost == max_ccu * $3. This seems like ti would be cost prohibitive for some usecases but work well for others.

In any case, the general idea does seem worth looking into more. Any other services you think we should check out would be good data points as well.

Re vendor lock in: Things are already broken down into an "adapter" interface for the WebRTC stuff (this is how we transitioned from Janus to MediaSoup, see https://github.com/mozilla/hubs/blob/master/src/naf-dialog-adapter.js. My bigger concern would be the complexity of trying to support multiple adapters, and the sort of feature lock in you would get by not having full control over the service. Basically if the feature doesn't exist in Chime and you want to add it, you either don't add the feature or have to wholesale switch to something else.

[rawnsley]

Thanks for the feedback. Yes, the pricing looks like a showstopper:

The Chime SDK is probably a better fit than the full Pro service, but it isn't much cheaper at $0.0017 / user / minute = $2.45 / user / day.

Comparing that to running our own SFU on a t3.2xlarge instance at $0.3341 / hour = $8.02 / day and can handle 12 rooms with 25 active concurrent users = 300 users. Therefore the cost would be $8.02 / 300 = $0.026 / user / day. That's a factor of 100 cheaper!

Also you might be right about flexibility: the SDK summary notes that audio is combined on the server and there is no way to modulate this per user (e.g. for spatialization). Perhaps you could host a "meeting" per user pair, but that increases the cost and may not scale on the client.

I'm guessing the price difference is due to the overhead of mixing the streams on the server side. This is worth considering as part of the larger discussion around this topic - it may not come for free.

It may still be useful to consider more granular scalability of the SFU service using Elastic Beanstalk or Lambda.

[gfodor]

This is another SaaS WebRTC SFU I've come across before. https://www.agora.io/en/

And yes I imagine hosting your own SFU is going to be much, much cheaper than any of these options. For elasticity it may be worth looking into AWS Fargate.

[gfodor]

Also there is not a consensus on the value of server side mixing - the webrtc term for this is MCU (vs SFU) https://bloggeek.me/webrtc-multiparty-video-alternatives/

In load testing, the SFU ends up being CPU bound on TLS encryption, so anything that adds CPU load to the WebRTC server is probably going to result in net cost increases.

[gfodor]

Another way to look at it is if you want to increase SFU capacity or reduce costs, figure out a way to get that encryption and decryption to happen somewhere else - might be possible with some hacks once insertable streams comes out and e2e starts to become the norm.

[rawnsley]

I guess if end-2-end encryption is ultimately a requirement then a "dumb pipe" SFU would be very efficient. On the plus side the hosting would be cheaper, the VoIP more secure, and the client mixing options more sophisticated (environment based acoustics etc...). On the negative side you are limited by the number of incoming steams a client can handle.

My guess is that this limit is probably larger than the number of incoming streams anyone would want to experience at any one time, so perhaps one answer is to dynamically manage the active connections in a room: closest avatars first with a bit of hysteresis to avoid ping-ponging.

Apologies if I'm stumbling over old ground - this is not my area of expertise!

[gfodor]

Yep, wrt e2e I don't think it will offload the TLS encryption from the server, but I'm not totally up to speed on it yet. I think it will just be double-encrypted, once with the usual DTLS keypair and then again with the peer-negotiated (secret) keypair.

[pattersonbl2]

Stale discussion question marking as answered.