Close Caption SEI messages in H.264/H.265 coded stream

[alabou]

For IS-04, video/smpte291 media_type is defined as a Flow of format urn:x-nmos:format:data. It is defined as an allowed media_type for the generic receiver_data.json schema and a the more specific flow_sdianc_data.json schema.

In general I would assume that if close caption data is available in an NMOS system, it will appear as a Source/Flow of urn:x-nmos:format:data format and media_type video/smpte291 or any other standard way of transporting such close caption data.

Supporting in-band close caption as SEI messages implies that the H.264/H.265 Flow is associated to a Source providing a video and a data Flow that result in the H.264/H.265 Flow with embedded close caption. The H.264/H.265 Flow then has a parent video Flow and a parent data Flow. It makes the model a little more complex but it seems acceptable.

                                              parent
               +---------- Flow_video/raw <-----+
               |                                |
               v              parent            |
      +-- Source_video <--------+               |
      |                         |               |
SDI --|                   Source_video <-- Flow_video/H264 <-- Sender
      |                         |               |
      +-- Source_data  <--------+               |
               ^              parent            |
               |                                |
               +----- Flow_data/smpte291 <------+
                                              parent

It seems that the embedding of the close caption information in SEI messages of the coded H.264/H.265 stream does not add anything to BCP-006-02 and BCP-006-03 as long as there is no specific information/control about how the embedding is performed. Simply looking at the coded Flow attributes does not indicate that there is such embedding. A Controller/User must look at the Flows/Sources parents relationship to realize that such embedding is performed.

[cristian-recoseanu]

A Controller/User must look at the Flows/Sources parents relationship to realize that such embedding is performed.

Wouldn't the main H264 flow be of urn:x-nmos:format:mux format if that were the case?

[garethsb]

Wouldn't the main H264 flow be of urn:x-nmos:format:mux format if that were the case?

I don't think that needs to be the case. The parents relation indicates this Flow is derived from the referenced Flows in some manner but doesn't indicate how. Perhaps we need an optional parent_roles attribute that allows an extensible description of how each parent affects a child Flow.