Skip to content

Commit 5d978d0

Browse files
committed
Add PTS and DTS sections
1 parent a62cba5 commit 5d978d0

1 file changed

Lines changed: 53 additions & 0 deletions

File tree

guides/useful_concepts/timestamps.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,56 @@ example video frames in raw video streams or RTP packets in RTP streams.
4141
These units finally can have timestamps assigned to them - and in
4242
most cases they do. For example, a PTS assigned to a buffer containing a
4343
raw video frame determines then the frame should be displayed.
44+
45+
## Presentation Time Stamps
46+
47+
As previously mentioned, PTSs are used to tell when a piece of media should be
48+
presented to the user. It can mean either displaying a video frame, or playing a
49+
chunk of audio.
50+
51+
### Realtimer
52+
53+
[Realtimer](https://hexdocs.pm/membrane_realtimer_plugin/Membrane.Realtimer.html)
54+
is an element from
55+
[membrane_realtimer_plugin](https://hex.pm/packages/membrane_realtimer_plugin).
56+
It takes in a stream and limits it's flow according to it's PTSs. For example, if
57+
it receives three buffers with `:pts` of 0ms, 200ms and 400ms, then it will send
58+
the first buffer, the second buffer after 200ms pass, and the third one after
59+
another 200ms pass.
60+
61+
This element is useful if we have non-realtime input, and realtime output, for
62+
example we want to stream the contents of a MP4 file with WebRTC. If we didn't
63+
use Realtimer, then we would read contents of the file as fast as possible and
64+
send them as fast as possible, which is not what we want, we want the receiver
65+
to receive the stream in realtime, so that they can display it as it comes.
66+
67+
## Decoding Time Stamps
68+
69+
The purpose of DTSs is to tell a decoder when a given media chunk should be
70+
decoded. In a lot of codecs the media can be decoded as it comes, but in some
71+
cases, like in [H264](../membrane_tutorials/h264/1_Introduction.md), it's
72+
not that simple. In a nutshell, in H264 some frames are encoded based on
73+
information from other frames. There are three main types of frames:
74+
75+
* I-frame (Intra-coded picture) - A frame of this type is encoded without the
76+
information from any other frames.
77+
* P-frame (Predicted picture) - A frame of this type is encoded with the usage
78+
of information of previous frames. If we have a static shot, then it takes
79+
much less space to encode a frame by using the fact that it's almost the
80+
same as the previous one.
81+
* B-frame (Bidirectional predicted picture) - A frame of this type is similar
82+
to a P-frame, as it uses information from other frames for it's encoding.
83+
However, it not only depends on previous pictures, but also on future ones.
84+
That's where the DTSs come in, because to decode a B-frame we also need to
85+
decode all frames it's encoding is based on, including the future ones.
86+
For example, let's assume that we have a slice of a stream consisting of
87+
three frames where frame 2 is a B-frame that's encoded based on the
88+
frames 1 and 3. If a decoder receives these frames with the following
89+
timestamps:
90+
91+
1) pts: 0ms, dts: 0ms
92+
2) pts: 200ms, dts: 400ms
93+
3) pts: 400ms, dts: 200ms
94+
95+
It will first decode the frames in order (1, 3, 2). If it hadn't decoded
96+
frames 1 and 3 first, it couldn't decode frame 2.

0 commit comments

Comments
 (0)