Skip to content

Commit 37201f6

Browse files
authored
Update RTSP tutorial (#101)
* Tutorials update WIP * Use the simplified pipeline * Fix numbering * Apply reviewers suggestions * Apply reviewers suggestions
1 parent 1ecc2ad commit 37201f6

11 files changed

+90
-205
lines changed
File renamed without changes.
Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,15 @@
11
# Architecture
22

33
Now let's discuss how the architecture of our solution will look like.
4-
It will be a little different from the RTMP to HLS architecture.
5-
The main component will be the pipeline, which will ingest RTP stream and convert it to HLS. Beyond that we will also need a Connection Manager, which will be responsible for establishing an RTSP connection with the server.
4+
It will be a little different from the RTMP to HLS architecture. In most cases communication with a RTSP server is split into two phases:
65

7-
![image](assets/rtsp_architecture.drawio.png)
6+
- Negotiation of the stream parameters over RTSP.
7+
- Receiving RTP stream(s) that the client and server have agreed upon.
88

9-
When initializing, the pipeline will start a Connection Manager which starts an RTSP connection with the server. Once the connection is fully established, the pipeline will be notified.
9+
Both of these phases are handled by RTSP Source. Let's take a closer look how each of them folds out:
1010

11-
Let's take a closer look on each of those components:
12-
13-
## Connection Manager
14-
The role of the connection manager is to initialize RTSP session and start playing the stream.
11+
## Establishing the connection
12+
When establishing a connection the source will act as a connection manager, initializing the RTSP session and starting the stream playback.
1513
It communicates with the server using the [RTSP requests](https://antmedia.io/rtsp-explained-what-is-rtsp-how-it-works/#RTSP_requests). In fact, we won't need many requests to start playing the stream - take a look at the desired message flow:
1614

1715
![image](assets/connection_manager.drawio.png)
@@ -20,18 +18,21 @@ First we want to get the details of the video we will be playing, by sending the
2018
Then we call the `SETUP` method, defining the transport protocol (RTP) and client port used for receiving the stream.
2119
Now we can start the stream using `PLAY` method.
2220

23-
## Pipeline
21+
## Receiving the stream
2422

25-
The pipeline consists of a couple elements, each of them performing a specific media processing task. You can definitely notice some similarities to the pipeline described in the [RTMP architecture](02_RTMP_Introduction.md). However, we will only be processing video so only the video processing elements will be necessary.
23+
The source is a bin containing a few elements, each of them performing a specific media processing task. You can definitely notice some similarities to the pipeline described in the [RTMP architecture](03_RTMP_Architecture.md). However, we will only be processing video so only the video processing elements will be necessary.
2624

2725
![image](assets/rtsp_pipeline.drawio.png)
2826

29-
We have already used the, `H264 Parser`, `MP4 H264 Payloader`, `CMAF Muxer` and `HLS Sink` elements in the RTMP pipeline, take a look at the [RTMP to HLS architecture](03_RTMP_SystemArchitecture.md) chapter for details of the purpose of those elements.
27+
We have already used the `H264 Parser` and `HLS Sink Bin` elements in the RTMP pipeline, take a look at the [RTMP to HLS architecture](03_RTMP_Architecture.md) chapter for details of the purpose of those elements.
3028

3129
Let us describe briefly what is the purpose of the other components:
3230

3331
### UDP Source
3432
This element is quite simple - it receives UDP packets from the network and sends their payloads to the next element.
3533

36-
### RTP SessionBin
37-
RTP SessionBin is a Membrane's Bin, which is a Membrane's container used for creating reusable groups of elements. In our case the Bin handles the RTP session with the server, which has been set up by the Connection Manager.
34+
### RTP Demuxer
35+
This element is responsible for getting media packets out of the RTP packets they were transported in and routing them according to their [SSRC](https://datatracker.ietf.org/doc/html/rfc3550#section-3). In our case we only receive a single video stream, so only one output will be used.
36+
37+
### RTP H264 Depayloader
38+
When transporting H264 streams over RTP they need to be split into chunks and have some additional metadata included. This element's role is to unpack the RTP packets it receives from the Demuxer into a pure H264 stream that can be processed further.
Lines changed: 15 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,43 @@
11
In the tutorial we won't explain how to implement the solution from the ground up - instead, we will run the existing code from [Membrane demos](https://github.com/membraneframework/membrane_demo).
22

33
To run the RTSP to HLS converter first clone the demos repo:
4-
```console
4+
```bash
55
git clone https://github.com/membraneframework/membrane_demo.git
66
```
77

8-
```console
8+
```bash
99
cd membrane_demo/rtsp_to_hls
1010
```
1111

1212
Install the dependencies
13-
```console
13+
```bash
1414
mix deps.get
1515
```
1616

17-
Make sure you have those libraries installed as well:
18-
- gcc
19-
- libc-dev
20-
- ffmpeg
21-
22-
On ubuntu:
23-
```console
24-
apt-get install gcc libc-dev ffmpeg
25-
```
26-
2717
Take a look inside the `lib/application.ex` file. It's responsible for starting the pipeline.
2818
We need to give a few arguments to the pipeline:
2919
```elixir
30-
@rtsp_stream_url "rtsp://rtsp.membrane.work:554/testsrc.264"
31-
@output_path "hls_output"
32-
@rtp_port 20000
20+
rtsp_stream_url = "rtsp://localhost:30001"
21+
output_path = "hls_output"
22+
rtp_port = 20000
3323
```
3424

35-
The `@output_path` attribute defines the storage directory for hls files and the `@rtp_port` defines on which port we will be expecting the rtp stream, once the RTSP connection is established.
25+
The `output_path` attribute defines the storage directory for hls files and the `rtp_port` defines on which port we will be expecting the rtp stream, once the RTSP connection is established.
3626

37-
The `@rtsp_stream_url` attribute contains the address of the stream, which we will be converting. It is a sample stream prepared for the purpose of the demo.
27+
The `rtsp_stream_url` attribute contains the address of the stream, which we will be converting. If you want to receive a stream from some accessible RTSP server, you can pass it's URL here. In this demo we'll run our own, simple server, using port 30001:
28+
29+
```bash
30+
mix run server.exs
31+
```
3832

3933
Now we can start the application:
40-
```console
34+
```bash
4135
mix run --no-halt
4236
```
4337

44-
The pipeline will start playing, after a couple of seconds the HLS files should appear in the `@output_path` directory. In order to play the stream we need to first serve them. We can do it using simple python server.
45-
46-
```console
47-
python3 -m http.server 8000
48-
```
38+
The pipeline will start playing, after a couple of seconds the HLS files should appear in the `@output_path` directory.
4939

5040
Then we can play the stream using [ffmpeg](https://ffmpeg.org/), by pointing to the location of the manifest file:
51-
```console
41+
```bash
5242
ffplay http://YOUR_MACHINE_IP:8000/rtsp_to_hls/hls_output/index.m3u8
5343
```

broadcasting/10_ConnectionManager.md

Lines changed: 0 additions & 71 deletions
This file was deleted.

broadcasting/10_RTSP_Pipeline.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
As explained in the [Architecture chapter](08_RTSP_Architecture.md), the pipeline will consist of an RTSP Source and an HLS Sink Bin. For now we won't connect this elements in any way, since we don't have information about what tracks we'll receive from the RTSP server which we're connecting to.
2+
3+
##### lib/pipeline.ex
4+
```elixir
5+
@impl true
6+
def handle_init(_context, options) do
7+
spec = [
8+
child(:source, %Membrane.RTSP.Source{
9+
transport: {:udp, options.port, options.port + options.port_range},
10+
allowed_media_types: [:video, :audio],
11+
stream_uri: options.stream_url,
12+
on_connection_closed: :send_eos
13+
}),
14+
child(:hls, %Membrane.HTTPAdaptiveStream.SinkBin{
15+
target_window_duration: Membrane.Time.seconds(120),
16+
manifest_module: Membrane.HTTPAdaptiveStream.HLS,
17+
storage: %Membrane.HTTPAdaptiveStream.Storages.FileStorage{
18+
directory: options.output_path
19+
}
20+
})
21+
]
22+
23+
{[spec: spec], %{parent_pid: options.parent_pid}}
24+
end
25+
```
26+
27+
Once we receive the `{:set_up_tracks, tracks}` notification from the source we have the information what tracks have been set up during connection establishment and what we should expect. First we filter these tracks, so that we have at most one video and audio track each. Then we can create specs that will connect output pads of the source with input pads of the sink appropriately - audio to audio and video to video.
28+
29+
##### lib/pipeline.ex
30+
```elixir
31+
@impl true
32+
def handle_child_notification({:set_up_tracks, tracks}, :source, _ctx, state) do
33+
track_specs =
34+
Enum.uniq_by(tracks, & &1.type)
35+
|> Enum.filter(&(&1.type in [:audio, :video]))
36+
|> Enum.map(fn track ->
37+
encoding =
38+
case track do
39+
%{type: :audio} -> :AAC
40+
%{type: :video} -> :H264
41+
end
42+
43+
get_child(:source)
44+
|> via_out(Pad.ref(:output, track.control_path))
45+
|> via_in(:input,
46+
options: [encoding: encoding, segment_duration: Membrane.Time.seconds(4)]
47+
)
48+
|> get_child(:hls)
49+
end)
50+
51+
{[spec: track_specs], state}
52+
end
53+
```
54+
55+
By doing this we are prepared to receive the streams when a `PLAY` request is eventually sent by the source and the server starts streaming.

broadcasting/11_RTSP_Pipeline.md

Lines changed: 0 additions & 89 deletions
This file was deleted.
File renamed without changes.
File renamed without changes.
-86.9 KB
Loading

0 commit comments

Comments
 (0)