@@ -41,3 +41,56 @@ example video frames in raw video streams or RTP packets in RTP streams.
4141These units finally can have timestamps assigned to them - and in
4242most cases they do. For example, a PTS assigned to a buffer containing a
4343raw video frame determines then the frame should be displayed.
44+
45+ ## Presentation Time Stamps
46+
47+ As previously mentioned, PTSs are used to tell when a piece of media should be
48+ presented to the user. It can mean either displaying a video frame, or playing a
49+ chunk of audio.
50+
51+ ### Realtimer
52+
53+ [ Realtimer] ( https://hexdocs.pm/membrane_realtimer_plugin/Membrane.Realtimer.html )
54+ is an element from
55+ [ membrane_realtimer_plugin] ( https://hex.pm/packages/membrane_realtimer_plugin ) .
56+ It takes in a stream and limits it's flow according to it's PTSs. For example, if
57+ it receives three buffers with ` :pts ` of 0ms, 200ms and 400ms, then it will send
58+ the first buffer, the second buffer after 200ms pass, and the third one after
59+ another 200ms pass.
60+
61+ This element is useful if we have non-realtime input, and realtime output, for
62+ example we want to stream the contents of a MP4 file with WebRTC. If we didn't
63+ use Realtimer, then we would read contents of the file as fast as possible and
64+ send them as fast as possible, which is not what we want, we want the receiver
65+ to receive the stream in realtime, so that they can display it as it comes.
66+
67+ ## Decoding Time Stamps
68+
69+ The purpose of DTSs is to tell a decoder when a given media chunk should be
70+ decoded. In a lot of codecs the media can be decoded as it comes, but in some
71+ cases, like in [ H264] ( ../membrane_tutorials/h264/1_Introduction.md ) , it's
72+ not that simple. In a nutshell, in H264 some frames are encoded based on
73+ information from other frames. There are three main types of frames:
74+
75+ * I-frame (Intra-coded picture) - A frame of this type is encoded without the
76+ information from any other frames.
77+ * P-frame (Predicted picture) - A frame of this type is encoded with the usage
78+ of information of previous frames. If we have a static shot, then it takes
79+ much less space to encode a frame by using the fact that it's almost the
80+ same as the previous one.
81+ * B-frame (Bidirectional predicted picture) - A frame of this type is similar
82+ to a P-frame, as it uses information from other frames for it's encoding.
83+ However, it not only depends on previous pictures, but also on future ones.
84+ That's where the DTSs come in, because to decode a B-frame we also need to
85+ decode all frames it's encoding is based on, including the future ones.
86+ For example, let's assume that we have a slice of a stream consisting of
87+ three frames where frame 2 is a B-frame that's encoded based on the
88+ frames 1 and 3. If a decoder receives these frames with the following
89+ timestamps:
90+
91+ 1 ) pts: 0ms, dts: 0ms
92+ 2 ) pts: 200ms, dts: 400ms
93+ 3 ) pts: 400ms, dts: 200ms
94+
95+ It will first decode the frames in order (1, 3, 2). If it hadn't decoded
96+ frames 1 and 3 first, it couldn't decode frame 2.
0 commit comments