Upload RTMP to YT - closing after a couple of buffers (NAL units)

[neilyoung]

This is probably not related to some malfunction of the lib. I'm just asking for similar experiences. I'm wondering, if some RTMP servers (namely Youtube) would need to have some special packets before the real video can flow. Especially I'm thinking about SPS and PPS units.

Look at this sequence: I'm getting NAL units from another source and forwarding it to the lib. Even though I'm sending video, the connection is torn down after a couple of ms (because of missing meta data?). You see that I'm just sending type 0x21 and 0x09 (the latter is ignored):

03-28 18:24:21.055 20872 20872 D: Opening RTMP 1280 720
03-28 18:24:21.118 20872 21033 D: Client connected true
03-28 18:24:21.246 20872 21033 D: Send NAL buffer, len 46058 (0 0 0 1 21 e0) -> 46078
03-28 18:24:21.252 20872 21033 D: Send NAL buffer, len 6 (0 0 0 1 9 10) -> 0
03-28 18:24:21.260 20872 21033 D: Send NAL buffer, len 45802 (0 0 0 1 21 e0) -> 45822
03-28 18:24:21.263 20872 21033 D: Send NAL buffer, len 6 (0 0 0 1 9 10) -> 0
03-28 18:24:21.268 20872 21033 D: Send NAL buffer, len 46259 (0 0 0 1 21 e0)  -> 46279
03-28 18:24:21.272 20872 21033 D: Send NAL buffer, len 6 (0 0 0 1 9 10) -> 0
03-28 18:24:21.279 20872 21033 D: Send NAL buffer, len 46638 (0 0 0 1 21 e0) -> 46658
03-28 18:24:21.285 20872 21033 D: Send NAL buffer, len 6 (0 0 0 1 9 10) -> 0
03-28 18:24:21.293 20872 21033 D: Send NAL buffer, len 46617 (0 0 0 1 21 e0) -> -20
03-28 18:24:21.295 20872 21033 D: Client connected false

I can see that my provided packet length is increased by 20 (dec) on return of the RTMPMuxer.writeVideo() routine until it is returning "-20" and reporting the disconnection.

Any pointer welcome.

[neilyoung]

No, it seems, this missing order is not the problem. What I can tell is, that the writeVideo routine is somehow sending garbage.

I have now replaced the YT endpoint by a local RTMP server, provided by node-media-server. As with like the YT endpoint the only thing, what works is the initial connection establishment.

The node-media-server complains literally about each incoming packet and Wireshark also doesn't like the packets much:

[NodeEvent on doneConnect] id=F2NUY8J4 args={"app":"live","type":"nonprivate","tcUrl":"rtmp://192.168.188.53:1935/live"}
29.3.2022 12:46:49 91076 [ERROR] rtmp packet parse error. {
  header: {
    fmt: 0,
    cid: 50,
    timestamp: 3078751,
    length: 1012450,
    type: 44,
    stream_id: 79653137
  },
  clock: 0,
  payload: null,
  capacity: 0,
  bytes: 0
}
29.3.2022 12:46:49 91076 [ERROR] rtmp packet parse error. {
  header: {
    fmt: 0,
    cid: 49,
    timestamp: 13436787,
    length: 3644398,
    type: 221,
    stream_id: 3637163481
  },
  clock: 0,
  payload: null,
  capacity: 0,
  bytes: 0
}
29.3.2022 12:46:49 91076 [ERROR] rtmp packet parse error. {
  header: {
    fmt: 0,
    cid: 17,
    timestamp: 14783451,
    length: 11984760,
    type: 214,
    stream_id: 2566289446
  },
  clock: 0,
  payload: null,
  capacity: 0,
  bytes: 0
}

On the other hand, at my source side, everything looks fine after NAL unit splitting. I'm using writeVideo then to send down the NAL units.

The packet contents shortly before providing it to writeVideo looks ok so far: NAL unit flag 00000001 followed by type and the rest of the trace doesn't look too bad

03-29 12:46:48.588  6116  6116 D: Opening RTMP 1280 720
03-29 12:46:48.790  6116  6279 D: Send NAL buffer, len 42121, type 0x21, first bytes: 00 00 00 01 21 e0 02 62 13 94 -> 42141
// Note: The next trace also shows the first few bytes of the following packet, because it is already in the buffer
03-29 12:46:48.793  6116  6279 D: Send NAL buffer, len 6, type 0x9, first bytes: 00 00 00 01 09 10 00 00 00 01 -> 0
03-29 12:46:48.797  6116  6279 D: Send NAL buffer, len 42607, type 0x21, first bytes: 00 00 00 01 21 e0 02 82 13 94 -> 42627
03-29 12:46:48.801  6116  6279 D: Send NAL buffer, len 6, type 0x9, first bytes: 00 00 00 01 09 10 00 00 00 01 -> 0
03-29 12:46:48.806  6116  6279 D: Send NAL buffer, len 44646, type 0x21, first bytes: 00 00 00 01 21 e0 02 a2 13 94 -> -20
03-29 12:46:48.806  6116  6279 D: Client connected false

EDIT: The value after the -> is the result of writeVideo.

So, I'm pretty convinced, that the NAL unit splitter does work correctly, the lib is fed with perfect NAL units. But it doesn't work somehow.

The H.264 source is a DJI and seems to be OK, since I can decode it perfectly to YUV and display on a surfaceview.

What is wrong here?

[neilyoung]

To continue the discussion here: I have patched the rtmp-lib a bit, namely I removed the requirement to have an SPS followed by a NAL unit. It seems, the whole thing is only working if the very first packet is an SPS. At least the RTMP_Write now doesn't complain anymore.

I'm not having any video on the remote side yet, but at least I have no errors anymore...

[neilyoung]

Good. Found the problem:

1. The lib expects to see SPS and PPS as ONE BUFFER.
2. The SPS-PPS buffer needs to the first buffer sent, before any ID frame or video slice can be sent.

Works with node-media-server and VLC as client. Unfortunately not yet with Youtube :(

[Gasol]

@neilyoung Which version are you using?

[neilyoung]

Don’t remember. Found another solution. Thanks

[Gasol]

@neilyoung I also have the same problem as publish stream to the platform like YouTube with the new release version 3.2.0. But it's okay on the others (e.g. srs). Do you remember what one did you use? The connection doesn't drop by the server If I downgrade to the old one 3.1.0.

The 3.2.0 release have renamed the package name from net.butterflytv to io.antmedia.

[neilyoung]

I’m sorry. I think I was using something else finally. And the problem with YT was: YT requires audio, even if muted.