SRT streaming to cloudflare: can't see live input?

[davidbaraff]

This is similar to the problem I had with castr, in that the Example application, which uses RP and MediaMixer works correctly with cloudflare, but my app (which is now working with castr after the recent update to allow me to set the expected media before publish), does not work with cloudflare.

I can stream RTMP to cloudflare with my app just fine, and see the live preview, and see the VOD when done.

When I stream SRT to cloudflare, I can see the VOD when the recording is done, but whenever I try to playback from cloudflare live when streaming, I get initially the current frame of the live stream, and then their player just spins. GPT suggests its an issue with IDR?

Anyway, my question is, what is different about the example app, which uses MediaMixer than what I'm doing, which is I construct a standard AVCaptureSession and in the output callbacks, feed audo/video samples as appropriate into the SRTStream.

Given that this doesn't work, if I make my own AVCaptureSession (and I have to be in control of the callback, as I have more processing I need to do there), is there some way I can make use of MediaMixer to feed it samples after I've processed them? (E.g. I take the CMSampleBuffer from the capture callback and heavily process it before sending it on). That's why I haven't simply attached audio/video as in the Example app; I have to get in in the middle.

But given that the example app allows live playback of SRT on cloudflare, and my app doesn't, there must be something extra that all the MediaMixer code is doing that I don't...

Any clues appreciated!

[shogo4405]

If there is a delay between publish and the first append, a similar issue may occur.
Even a short interval, such as two seconds, can cause this. If you are calling AVCaptureSession.startRunning() after publish, you might want to try starting it beforehand.

HaishinKit also provide an option in MediaMixer for using your own AVCaptureSession:

// Start MediaMixer in manual mode.
let mixer = MediaMixer(captureSessionMode: .manual)
await mixer.startRunning()

// Manually append each CMSampleBuffer.
await mixer.append(sampleBuffer)

[davidbaraff]

Thank you, using the mixer worked perfectly. Now I can ditch my audio conversion code too. Thank you so much!

If I ever hear back from youtube, I'll let you know if SRT streaming to their platform works using this method.

[davidbaraff]

There is no delay. My capture session is up and running long before I call publish so samples would be sent right away.But if I can manually call mixer.append(sampleBuffer) then I will try that.1.  I assume that works for both audio and video samples? 2. If the mixer is setup, but I have not yet called publish, I assume calling mixer.append() is low-cost?  (I don’t want to run expensive conversions before the stream has started, as we run the capture session all the time, even when not streaming.)Indeed, that would be preferable, because when calling append() directly on the stream instead, I had to write code that would do conversion of CMSamples to AVAudio PCM data, which I would greatly prefer not to write myself.I’ll try that and let you know if it worked. Sent from my iPadOn Nov 25, 2025, at 7:23 AM, shogo4405 ***@***.***> wrote: If there is a delay between publish and the first append, a similar issue may occur. Even a short interval, such as two seconds, can cause this. If you are calling AVCaptureSession.startRunning() after publish, you might want to try starting it beforehand. HaishinKit also provide an option in MediaMixer for using your own AVCaptureSession: // Start MediaMixer in manual mode. let mixer = MediaMixer(captureSessionMode: .manual) await mixer.startRunning() // Manually append each CMSampleBuffer. await mixer.append(sampleBuffer) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[shogo4405]

Yes. The processing related to mixer.append() is designed to be lightweight.
Since I believe you are not using the video compositing features built into MediaMixer, you can lower the overhead even further by doing the following, which will bypass unnecessary processing.

var videoSetting = await mixer.videoMixerSettings
videoSetting.mode = .passthrough
await mixer.setVideoMixerSettings(videoSetting)

[davidbaraff]

Thanks. I've added that, though it already was on passthrough.

Just in case it helps someone else, here's the sequence of calls I use to reliably stream to SRT, using my own AVCaptureSession. I should note that the capture session is already setup and running before I decide to start streaming, but here is how I start my stream now.

        let connection = SRTConnection()
        let stream = SRTStream(connection: connection)
        let mixer = MediaMixer(captureSessionMode: .manual, multiTrackAudioMixingEnabled: false)

        try await mixer.setFrameRate(60)  
        var videoSetting = await mixer.videoMixerSettings
        videoSetting.mode = .passthrough
        await mixer.setVideoMixerSettings(videoSetting)
        
        await mixer.startRunning()
        await mixer.addOutput(stream)

        try await stream.setAudioSettings(audioSettings)
        try await stream.setVideoSettings(videoSettings)
        _ = try await connection.connect(ingestURL)

        let connected = await connection.connected
        guard connected else {
            /* complain */
            return
        }

        await stream.setExpectedMedias([.audio, .video])
        await stream.publish()

        self.mediaMixer = mixer

---

My AVCaptureCallback calls
self.mediaMixer?.append(sampleBuffer)

so prior to starting the stream, this is a no-op. As soon as the stream has started though, samples begin to be delivered. RTMP is far more forgiving, but pretty much the exact above sequence was necessary to get things going correct with castr, cloudflare and (hopefully someday) youtube.

[davidbaraff]

A final note: I was given access again yesterday to a youtube SRT ingest for evaluation; as of now, we’ve been streaming 14 hours without a hitch, so all the changes and suggestions have worked out perfectly. Thank you again for the help.

[shogo4405]

Thank you very much for your courteous message. I'm glad to hear that it worked well.
Once it becomes generally available, I would also like to test it myself.