max_bit_rate in RTP parameters should actually be min_bit_rate?

[adriancable]

Right now when starting a video stream, HAP-NodeJS passes something like this to handleStreamRequest on my delegate:

{
  sessionID: 'd06ea903-4ac4-4029-97e9-2e57db58b7d6',
  type: 'start',
  video: {
    codec: 0,
    profile: 2,
    level: 2,
    packetizationMode: 0,
    cvoId: undefined,
    width: 1280,
    height: 720,
    fps: 30,
    pt: 99,
    ssrc: 1947509574,
    max_bit_rate: 299,
    rtcp_interval: 0.5,
    mtu: 1378
  },
  audio: { ... }
}

Note max_bit_rate and indeed, at least in the very old HomeKit docs from Apple I have, this is what that says, too. But I don't think it's right. I think it actually should be min_bit_rate. Looking at the console output from homed on iOS, I see this:

[Garden Camera/74/D06EA903-4AC4-4029-97E9-2E57DB58B7D6/kDefaultCameraApplicationIdentifier] Writing start stream configuration: 
    {
          sessionControl = 
        {
              tlvDatablob = (null) 
              controlCommand = HMDSessionControlCommandStart 
              sessionID = D06EA903-4AC4-4029-97E9-2E57DB58B7D6 
        } 
          videoParameters = 
        {
              tlvDatablob = (null) 
              videocodec = HMDVideoCodecTypeH264 
              codecParameters = 
            {
                  tlvDatablob = (null) 
                  h264Profile = 
                  [ 
                {
                     h264Profile = HMDH264ProfileTypeHigh
                }
                  ] 
                  levels = 
                  [ 
                {
                     h264Level = HMDH264LevelType4
                }
                  ]
                  packetizationModes = 
                  [ 
                {
                     packetizationMode = HMDPacketizationModeTypeSingleNonInterleaved
                }
                  ]
            } 
              attributes = 
            {
                  tlvDatablob = (null) 
                  imageWidth = 1280 
                  imageHeight = 720 
                  resolution = 
                {
                     resolutionType = HMDVideoResolutionType1280x720
                } 
                  framerate = 30 
            } 
              rtpParameters = 
            {
                  tlvDatablob = (null) 
                  syncSource = 1947509574 
                  payloadType = 99 
                  minimumBitrate = 299000 
                  maximumBitrate = 1078000 
                  rtcpInterval = 0.5 
                  maxMTU = 1378 
                  comfortNoisePayloadType = (null) 
            } 
        } 
          audioParameters = { ... }
    }

Note that what HAP-NodeJS returns as max_bit_rate matches minimumBitrate here (299000), not maximumBitrate (1078000).

Also, I did an experiment: if I do send RTP data faster than what's called maximumBitrate in the homed logs for a few seconds, homed will kill the stream. So it does really appear that this is the enforced maximumBitrate, i.e. the iOS logs from homed are correct and HAP-NodeJS is wrong.

What I haven't looked at: how we get at the maximumBitrate from the RTP stream configuration TLV data sent by homed. This would probably be very useful for plug-ins to know. Right now since plug-ins are interpreting what's actually the min bite rate as the max bit rate, camera plug-ins are usually universally sending much lower quality video to HomeKit than they should be, which is quite sad.

[n0rt0nthec4t]

Following what happens with this, as have noticed also

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bauer-andreas]

Did you uncover any further information on this? I have never read anything in the documentation about a "minimum bitrate". All docs always refer to a maximum bitrate?
How do certified accessories behave?

[adriancable]

@Supereg - I'm not sure if it's possible to answer that. It would require reverse engineering the certified accessory to see how it interprets that value.

One other point, though. As well as the iOS logs themselves referring to that value as the minimum bitrate (and also providing another, much higher value which is referred to as maximum bitrate), it really doesn't seem plausible to interpret that value as a maximum bitrate, despite what the docs say.

The numbers are often extremely small, implausibly low for a sensible bitrate. For example we frequently see that value being provided by HomeKit as 132000. (Usually, the flow is that the 'start' request has 299000, and then shortly afterwards a 'reconfigure' request with 132000.) 132kbit/sec just isn't a plausible maximum bitrate for H264 video, at 640 x 360 or 1280 x 720 resolution. It's too low to give anything like acceptable quality video. This also suggests that the docs are wrong (and the iOS logs are right).

I do note that the non-commercial HomeKit spec docs from Apple that are 'floating around' are very very old, and it's quite possible this has been corrected in newer versions. Unfortunately, I don't have access to (and do not want access to) the 'commercial' up-to-date docs so have no way of checking.

[bauer-andreas]

@adriancable have you monitored what iOS sends in future reconfigure stream requests? Maybe they start with the minimum bitrate as the „start“ maximum bitrate and iteratively increase the bitrate. The reconfigure stream request is something most plugins don’t really implement as there is no easy way to do it with ffmpeg (when using the command line interface).

[adriancable]

@Supereg - actually, it's the opposite. Usually, the 'reconfigure' request drops the bitrate.

For example, initially:

            {
                  tlvDatablob = (null) 
                  syncSource = 1947509574 
                  payloadType = 99 
                  minimumBitrate = 299000 
                  maximumBitrate = 1078000 
                  rtcpInterval = 0.5 
                  maxMTU = 1378 
                  comfortNoisePayloadType = (null) 
            }

Then 20 seconds later there's a 'reconfigure' request with a new minimumBitrate = 132000.