Skip to content

Commit e9b15ce

Browse files
committed
Add visualisation
1 parent 5d978d0 commit e9b15ce

2 files changed

Lines changed: 23 additions & 19 deletions

File tree

125 KB
Loading

guides/useful_concepts/timestamps.md

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Timestamps - the stamps in time
1+
# Timestamps
22

33
In a nutshell, timestamps determine when a given event occurred in time. For
44
example when you take a photo with your phone, the exact time and date the photo
@@ -61,36 +61,40 @@ another 200ms pass.
6161
This element is useful if we have non-realtime input, and realtime output, for
6262
example we want to stream the contents of a MP4 file with WebRTC. If we didn't
6363
use Realtimer, then we would read contents of the file as fast as possible and
64-
send them as fast as possible, which is not what we want, we want the receiver
65-
to receive the stream in realtime, so that they can display it as it comes.
64+
send them over as fast as possible, which is not something we want. We want the receiver
65+
to get the stream in realtime, so that they can display it as it comes.
6666

6767
## Decoding Time Stamps
6868

69-
The purpose of DTSs is to tell a decoder when a given media chunk should be
69+
The purpose of DTSs is to tell a decoder when a frame should be
7070
decoded. In a lot of codecs the media can be decoded as it comes, but in some
7171
cases, like in [H264](../membrane_tutorials/h264/1_Introduction.md), it's
7272
not that simple. In a nutshell, in H264 some frames are encoded based on
7373
information from other frames. There are three main types of frames:
7474

7575
* I-frame (Intra-coded picture) - A frame of this type is encoded without the
76-
information from any other frames.
76+
information from any other frames. Sometimes referred to as a _keyframe_.
7777
* P-frame (Predicted picture) - A frame of this type is encoded with the usage
78-
of information of previous frames. If we have a static shot, then it takes
78+
of information from previous frames. If we have a static scene, then it takes
7979
much less space to encode a frame by using the fact that it's almost the
80-
same as the previous one.
80+
same as the previous one and encoding only the things that have changed.
8181
* B-frame (Bidirectional predicted picture) - A frame of this type is similar
8282
to a P-frame, as it uses information from other frames for it's encoding.
8383
However, it not only depends on previous pictures, but also on future ones.
8484
That's where the DTSs come in, because to decode a B-frame we also need to
85-
decode all frames it's encoding is based on, including the future ones.
86-
For example, let's assume that we have a slice of a stream consisting of
87-
three frames where frame 2 is a B-frame that's encoded based on the
88-
frames 1 and 3. If a decoder receives these frames with the following
89-
timestamps:
90-
91-
1) pts: 0ms, dts: 0ms
92-
2) pts: 200ms, dts: 400ms
93-
3) pts: 400ms, dts: 200ms
94-
95-
It will first decode the frames in order (1, 3, 2). If it hadn't decoded
96-
frames 1 and 3 first, it couldn't decode frame 2.
85+
decode all frames it's based on, including the future ones.
86+
87+
![image](./assets/frame_types.png)
88+
89+
For example, let's assume that we have a slice of a stream from the diagram,
90+
consisting of four frames. Frames 1 and 4 are I-frames, frame 2 is a P-frame
91+
depending on frame 1, and frame 3 is a B-frame depending of frames 2 and 4.
92+
If a decoder receives these frames with the following timestamps:
93+
94+
1) pts: 0ms, dts: 0ms
95+
2) pts: 200ms, dts: 200ms
96+
3) pts: 400ms, dts: 600ms
97+
4) pts: 600ms, dts: 400ms
98+
99+
It will first decode the frames in order (1, 2, 4, 3), according to their DTS.
100+
If it hadn't decoded frames 2 and 4 first, it couldn't have decoded frame 3.

0 commit comments

Comments
 (0)