Audio Distortion/Noise When Streaming from DeckLink Hardware

[ekiny]

Environment:
HaishinKit version: (latest from Swift Package Manager)
Platform: macOS (Apple Silicon)
Swift version: 5.x
Audio source: Blackmagic DeckLink hardware capture card

Problem:
I'm experiencing audio distortion/noise when streaming audio from a DeckLink capture device through HaishinKit RTMP. The video works perfectly, but audio has constant crackling/static noise.

Audio Source Details:
Input Format: DeckLink SDK provides Int16 interleaved stereo PCM
Sample Rate: 48000 Hz
Channels: 2 (Stereo)
Samples per frame: ~2000 samples per callback
Data layout: [L, R, L, R, L, R, ...] (interleaved Int16)

Current Implementation:

// 1. AAC encoder configuration
     let audioSettings = AudioCodecSettings(
    bitRate: 128 * 1000,      // 128 kbps
    downmix: false,            // Keep stereo
    channelMap: nil,
    sampleRate: 48000.0,
    format: .aac
)
try await rtmpStream.setAudioSettings(audioSettings)

// 2. Converting DeckLink Int16 to AVAudioPCMBuffer
func appendAudioData(_ audioData: UnsafeRawPointer, sampleCount: UInt32, 
                     sampleRate: Double, channelCount: UInt32) {
    // Copy data immediately (pointer is temporary)
    let bytesPerSample: Int = 2
    let totalBytes = Int(sampleCount) * Int(channelCount) * bytesPerSample
    let audioCopy = Data(bytes: audioData, count: totalBytes)
    
    queue.async { [weak self, audioCopy] in
        guard let self = self else { return }
        audioCopy.withUnsafeBytes { (ptr: UnsafeRawBufferPointer) in
            guard let baseAddress = ptr.baseAddress else { return }
            
            // Create Float32 NON-interleaved format
            let audioFormat = AVAudioFormat(
                commonFormat: .pcmFormatFloat32,
                sampleRate: sampleRate,
                channels: AVAudioChannelCount(channelCount),
                interleaved: false  // Non-interleaved
            )
            
            let pcmBuffer = AVAudioPCMBuffer(
                pcmFormat: audioFormat, 
                frameCapacity: AVAudioFrameCount(sampleCount)
            )
            pcmBuffer.frameLength = AVAudioFrameCount(sampleCount)
            
            // Convert Int16 interleaved → Float32 non-interleaved
            let sourceInt16 = baseAddress.assumingMemoryBound(to: Int16.self)
            let leftChannel = pcmBuffer.floatChannelData![0]
            let rightChannel = pcmBuffer.floatChannelData![1]
            
            for i in 0..<Int(sampleCount) {
                leftChannel[i] = Float(sourceInt16[i * 2]) / 32768.0
                rightChannel[i] = Float(sourceInt16[i * 2 + 1]) / 32768.0
            }
            
            // Send with cumulative timestamp
            let audioTime = AVAudioTime(
                sampleTime: AVAudioFramePosition(self.totalSamplesSent),
                atRate: sampleRate
            )
            self.totalSamplesSent += Int64(sampleCount)
            
            DispatchQueue.main.async {
                self.rtmpStream.append(pcmBuffer, when: audioTime)
            }
        }
    }
}

Questions:

Is the audio format conversion correct?
DeckLink sends Int16 interleaved → Should I convert to Float32 non-interleaved or Float32 interleaved?
Is interleaved: false correct for AVAudioFormat with HaishinKit's AAC encoder?

Is the de-interleaving logic correct?

   leftChannel[i] = sourceInt16[i * 2]      // Even indices (0,2,4...)
   rightChannel[i] = sourceInt16[i * 2 + 1]  // Odd indices (1,3,5...)

Is the timestamp calculation correct?

• Using cumulative sample count: AVAudioTime(sampleTime: totalSamplesSent, atRate: 48000)
• Incrementing by actual samples: totalSamplesSent += sampleCount

Using cumulative sample count:
AVAudioTime(sampleTime: totalSamplesSent, atRate: 48000)
Incrementing by actual samples: totalSamplesSent += sampleCount
Should I use rtmpStream.append(pcmBuffer, when: audioTime) or just rtmpStream.append(audioTime)?
Any known issues with DeckLink or hardware capture device audio?

Additional Info:
Video streaming works perfectly (using CMSampleBuffer from CVPixelBuffer)
Audio appears to be encoded (AAC packets are sent), but playback has severe distortion
The audio sounds like random noise/static, not just quiet crackling

Any help would be greatly appreciated!

Thank you! 🙏

[noho501]

I’m experiencing the same issue with the audio on iOS — it’s crackling during live streaming to YouTube.
I’m not sure how to debug this, but I noticed the following error appearing in the console:

<<< CMBufferQueue >>> signalled err=-12764 at <>:1838

Could this issue be related to that error?

My settings:

• Resolution: 1920×1080
• Frame rate: 30 fps
• Bitrate: 4000 kbps
• Mixer: .single
• Using startRecording
• Stream bitrate strategy enabled (StreamBitRateStrategy)

Please check the attached video — you can clearly hear the audio crackling issue there.

Hope someone can help with this! 🙏

IMG_0818.mov