diarization multiprocessing on CPUs

[MrEdwards007]

I am very interested in speaker-diarization but I do not have a supported GPU so this runs on my CPU.

I would like to run diarization on an chunks of a single input source, in parallel and stitch the results together.
Can this be done using the below methodology or is there another that would work?

All testing was done on a MacBook, macOS Big Sur using 2.3 GHz Intel Core i9, 16 cores, with 16G of RAM (GPU not supported)
My first target was to learn how long diarization would take under normal circumstances. I ran the process overnight.

My target (testing) file DonQuixote_OneHour is 3706.393 seconds - 01:01:46(H:M:S)
I guessed the process would be at most 1-1.5x the length of the audio on a CPU
The elapsed completion time for DonQuixote_OneHour_audio.rttm was 7:41:12 (H:M:S), which is greater than -7x (closer to -8x)

Once I established how long a typical process ran, I tried to learn if the was a signature]hash for each speaker identified, versus 'speaker 1, 2, 3, etc' If there is a signature, the process can be broken into chunks (I've already done this for transcription, including fixing the concatenated timelines) and run in parallel to bring the completion time down. If there is a signature\hash, each parallel process can be stitched back together, using the signature\hash of the speaker (assuming this would be the same across processes).

1. How would you advise accelerating on CPUs?
2. Is there an obtainable hash\signature for each identified user, that can be used across processes?
3. If thee is not a hash\signature for each identified user, can one be synthesized?

Below is the approach that I took with accelerating Whisper tasks, (specifically transcriptions)
openai/whisper#432

Using 011 of 16CPUs for the "tiny.en" model, a transcription speed (over real time) of 32.713x
Using 007 of 16CPUs for the "base.en" model, a transcription speed (over real time) of 16.416x
Using 009 of 16CPUs for the "small.en" model, a transcription speed (over real time) of 5.595x

Thank you for your time and assistance.

[Isuxiz]

Hello, I have similar ideas to you and came here through the search engine. May I ask if you finally found a good method? I would be very grateful if you share your solution!

[arcticloud]

You can chunk-and-parallelize, and on a 16-core CPU it will help — but there's one catch that makes the naive version give wrong results: diarization clustering is global. If you diarize each chunk independently, SPEAKER_00 in chunk 1 and SPEAKER_00 in chunk 3 are not guaranteed to be the same person, so you can't just concatenate the RTTMs. You have to re-map speakers across chunks by comparing embeddings (same idea as matching speakers across files).

A working pattern:

from concurrent.futures import ProcessPoolExecutor
# 1. split audio into N chunks with a small overlap (e.g. 5 min chunks, 10 s overlap)
# 2. diarize each chunk in its own process (ProcessPoolExecutor, one worker per core group)
# 3. extract one embedding per (chunk, local speaker)
# 4. cluster those embeddings globally -> consistent speaker IDs
# 5. stitch, using the overlap to resolve labels at chunk boundaries

Two practical notes:

• Load the pipeline inside each worker, not in the parent — the models don't pickle cleanly across processes.
• Set torch.set_num_threads(1) per worker so N processes don't oversubscribe the 16 cores and thrash.

Honestly, though: your ~7-8x realtime is normal for CPU on the older pipeline, and chunk-stitching adds real complexity and boundary errors. Before investing in it, try the newer community pipeline (noticeably faster) or a batched run on even a modest GPU — that's usually a bigger win than CPU parallelism for a single long file.