Feature request: audio stream

[crxporter]

If I understand correctly, some cameras have two-way audio. This plugin should work for audio coming from a camera playing with the RTSP stream.

What about if I have a camera with a speaker? It should be section 9.102 of the homekit spec describing audio from a microphone to a speaker. I don't actually currently have a camera with two-way audio but I might add one later.

It would be fun to talk to my wife and kids over the homekit camera!

---

Are others interested in this? Is this something that could be added sometime? Does it already work if we set up the right services?

Please don't rush anything, just wondering if it's part of hap node js.

[Shaquu]

About two way audio.
It looks like guys from hap-nodes has to solve it firstly.
I think someone mentioned here that it is possible to do with ffmpeg but can be choppy. Though not too many mentions of working two way audio.

About one way audio.
It is said in the discussion in the link above that one way audio should work good enough.
"You should use AAC-ELD" ~someone said.
So I would start with configuring ffmpeg to play audio and video from camera in VLC or any video player. Then we can move to HomeKit and see if configuration is ease or not. If not then I have to order some cheap camera supporting audio to test it.

[crxporter]

It seems like testing could be done by sending the audio stream buffer (from HomeKit) to a file then seeing if that audio file could play.

Honestly I don't even know if the camera I'm looking at (unifi g3 micro) can play audio from anything but the official Unifi system.

I currently only have cameras with one-way audio (camera to homekit) and it is working properly.

[crxporter]

Some interesting discussion about this here

Way above my understanding but seems like it’s at least possible.

[crxporter]

Just for fun tonight I added a linked "Speaker" service to one of my 3 cameras. Interestingly it added a microphone icon in the Home app to all 3 of the cameras - even though they are each on their own bridge. Also interesting, when I enabled/disabled the little microphone icon it would trigger the audio stream from the camera microphone (same as the speaker icon would normally).

I'd guess it all comes down with properly configuring the audio streams. Also seems like nobody else has spoken up asking for this so it's not something to spend time on at this point.

Maybe if we ever get to version 2.0 😉

[crxporter]

Closing - links are available to watch for future users/developers (future versions of ourselves?).

[crxporter]

I just scored a beta release unifi video doorbell. It's got two-way audio features. It should arrive in a week or so. I'll be working with everyone I can find to figure out the two way audio on it - I'm re-opening this issue to document my findings.

@Shaquu - any chance we'll get a beta release with the new hap-nodejs video features sometime soon? I'm not actually sure what changed there but I'd like to work with the latest and greatest...

[burnbrigther]

I hope you get some progress with this. 2-way audio with security cameras and homebridge has been a problem for a few years now.

[Shaquu]

@Shaquu - any chance we'll get a beta release with the new hap-nodejs video features sometime soon?

Yes!

Edit: npm i node-red-contrib-homekit-bridged@1.1.0-dev.6

[crxporter]

Awesome, I'll be setting up a dev environment for this - my doorbell should arrive tomorrow.

I'm also looking for help from this project, there are a couple of links there for a ring plugin that supports 2-way audio.

The question will be how it works with my particular camera... I don't know where or how to send the audio, that's another place things may get difficult.

[Shaquu]

I wanted to say that I will buy UniFi 2-way audio device to support you directly and then I have opened their site and saw that prices... Hah :)

I will try to contact them anyway.
Also do you have an access to UniFi protect?
If you want we can try to cooperate on that online to figure out how to plug it into nrchkb.

[crxporter]

Yeah, it's not a cheap system! I went all in early 2019 when my second "cheap" router in 6 months burned out. Then I've been collecting to the point where I now have way too much of their stuff. Oops.

I currently have a dream machine pro and 3 g3 flex cameras. They're all working just great in nrchkb. The dream machine runs UniFi protect. I'm polling the API every second to detect motion, it took me a little bit but it's been running for weeks without issues.

Here's a screenshot of a single camera data that I get back every second. I marked in red where I hope the doorbell might have helpful information for 2-way audio... We'll see tomorrow!

[crxporter]

Oh also for reference here's my current functional camera setup - still image, video, and audio all working.