Skip to content

Commit b7223f3

Browse files
NoarkhhNoarkhh
committed
Finish first version of the guide
1 parent 5fa0b6a commit b7223f3

1 file changed

Lines changed: 175 additions & 6 deletions

File tree

‎guides/useful_concepts/pads.md‎

Lines changed: 175 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,179 @@
11
# Everything about pads
22

3-
When developing some intuition about the structure of pipelines the pads are
3+
When developing intuition about the structure of pipelines pads are
44
something that can't be ignored. If you think about elements and bins as some
5-
sort of containers in which some processing happens, then pads are the parts
6-
that with these tanks are connected with. A pad of one element can only connect
7-
to a single pad of other element and only once two pads are connected the media
8-
can flow through them.
5+
sort of containers or boxes in which processing happens, then pads are the parts
6+
that with these containers are connected with. There are some constraints
7+
regarding pads:
98

10-
`t:Membrane.Pad.element_spec/0`
9+
* A pad of one element can only connect to a single pad of other element and
10+
only once two pads are connected communication through them can happen.
11+
* One pad needs to be an input pad, and the other an output pad.
12+
* The accepted stream formats of the pads need to match.
13+
14+
When looking at the insides of elements, the pads are their main way to
15+
communicate with other elements in the pipeline. When an element receives
16+
something from another one (e.g. a buffer or stream format), it receives it
17+
from a pad. The reference to this pad is then also available as an argument
18+
of the callback that handles the received thing. For example an invocation
19+
of the [following callback](`c:Membrane.Element.WithInputPads.handle_buffer/4`)
20+
would mean that a buffer `buffer` has arrived on a pad `some_pad`:
21+
22+
```elixir
23+
@impl true
24+
def handle_buffer(some_pad, buffer, context, state) do
25+
... ^^^^^^^^
26+
end
27+
```
28+
29+
When an element wants to send something to another element in the
30+
pipeline, most likely it should send it on a pad that's connected to it. It
31+
can do that by using the pad reference in actions that send things, for example
32+
returning the following [buffer action](`t:Membrane.Element.Action.buffer/0`)
33+
from a callback would mean that a buffer `buffer` will be sent on a pad `some_pad`:
34+
35+
```elixir
36+
@impl true
37+
def some_callback(...) do
38+
...
39+
{[buffer: {some_pad, buffer}], state}
40+
end ^^^^^^^^
41+
```
42+
43+
## Defining pads
44+
45+
To define what pads an element will have and how they'll behave we use
46+
[`def_input_pad/2`](`Membrane.Element.WithInputPads.def_input_pad/2`)
47+
and [`def_output_pad/2`](`Membrane.Element.WithOutputPads.def_output_pad/2`) macros.
48+
The first argument for these macros is a name, which then will be used to
49+
identify the pads. The second argument is a `t:Membrane.Pad.element_spec/0`
50+
keyword list, which is used to define how this pad will work. An option we'll
51+
now focus on is [`availability`](`t:Membrane.Pad.availability/0`), which
52+
determines if the pad is _static_ or _dynamic_.
53+
54+
### Static pads
55+
56+
Static pads are pretty straightforward - when a static pad is defined there
57+
will always be exactly one instance of this pad and it's referenced by it's
58+
name.
59+
60+
#### File Source Example
61+
62+
An example of element with only static pads is a [File Source](https://hexdocs.pm/membrane_file_plugin/Membrane.File.Source.html).
63+
This element reads contents of a file and sends them in batches through a static
64+
output pad. The content of the buffers sent by this element is unknown - the file
65+
that's being read can contain anything - so this pad has `:accepted_format` set to
66+
`%RemoteStream{type: :bytestream}`. That means that any stream format that
67+
matches on this struct can be sent on the output pad and this fact has to be
68+
accounted for when connecting an element after the source.
69+
70+
A pipeline spec with a file source passing buffers to a MP4 demuxer could look
71+
like this:
72+
73+
```elixir
74+
@impl true
75+
def handle_init(_context, state) do
76+
spec =
77+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
78+
|> via_out(:output)
79+
|> via_in(:input)
80+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
81+
82+
{[spec: spec], state}
83+
end
84+
```
85+
86+
This spec will connect a pad named `:output` of the source to a pad named
87+
`:input` of the demuxer. However this can be shortened - if an output pad is
88+
called `:output` or an input pad is called `:input`, their respective
89+
[`via_in/3`](`Membrane.ChildrenSpec.via_in/3`) and
90+
[`via_out/3`](`Membrane.ChildrenSpec.via_out/3`) calls can be omitted and
91+
Membrane will automatically recognize and connect them:
92+
93+
```elixir
94+
@impl true
95+
def handle_init(_context, state) do
96+
spec =
97+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
98+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
99+
100+
{[spec: spec], state}
101+
end
102+
```
103+
104+
### Dynamic pads
105+
106+
Dynamic pads are a bit more complex. They're used when the amount of pads of
107+
given type is variable - dependent on the processed stream or external factors.
108+
The creation of these pads is controlled by the parent of the element - if a
109+
[`:spec`](`t:Membrane.Pipeline.Action.spec/0`) action linking the dynamic pad is
110+
being executed, then the pad is created dynamically and the element needs to
111+
handle this, in most cases with
112+
[`handle_pad_added/3`](`c:Membrane.Element.Base.handle_pad_added/3`).
113+
114+
Another thing that's different are the pad references. The pad's name can't just
115+
be used as the pad's reference, because it wouldn't be unique. Dynamic pads are
116+
identified by [`Pad.ref/2`](`Membrane.Pad.ref/2`), that takes the pad's
117+
name and a unique reference as arguments. The result is a unique pad reference
118+
that is also associated with a given pad's specification through it's name.
119+
120+
#### MP4 Demuxer Example
121+
122+
An example of an element using dynamic pads is an
123+
[MP4 Demuxer](https://hexdocs.pm/membrane_mp4_plugin/Membrane.MP4.Demuxer.ISOM.html).
124+
This element has a input pad, from which it receives contents of a MP4
125+
container, and output pads, on which it'll send the different tracks
126+
that were in the container. The input pad can be static, however MP4 containers can
127+
have different numbers and kinds of tracks, so the output pad needs to be dynamic.
128+
129+
We'll consider the case when we don't have any prior information about the
130+
tracks in this MP4 container. Because of this, the parent pipeline or bin of this
131+
demuxer won't initially know how many pads should be connected. To solve this
132+
problem the demuxer will identify the tracks in the incoming stream and send a
133+
message to it's parent in the form of
134+
[`{:new_tracks, [{track_id :: integer(), content :: struct()}]}`](https://hexdocs.pm/membrane_mp4_plugin/Membrane.MP4.Demuxer.ISOM.html#t:new_tracks_t/0).
135+
The list contains a list of tuples corresponding to tracks, where the first
136+
element is a track id and will be used to identify corresponding pad,
137+
and the second a stream format contained in the track.
138+
139+
Initially the parent will only create the elements before and including the
140+
demuxer:
141+
142+
```elixir
143+
@impl true
144+
def handle_init(_context, state) do
145+
spec =
146+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
147+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
148+
149+
{[spec: spec], state}
150+
end
151+
```
152+
153+
The source will start providing the demuxer the MP4 container content,
154+
from which the demuxer will identify tracks and notify it's parent
155+
about them. The parent now has to connect an output pad of the demuxer for each
156+
track received, which can look like this:
157+
158+
```elixir
159+
@impl true
160+
def handle_child_notification({:new_tracks, tracks}, :mp4_demuxer, _context, state) do
161+
spec =
162+
Enum.map(tracks, fn {id, format} ->
163+
get_child(:mp4_demuxer)
164+
|> via_out(Pad.ref(:output, id))
165+
|> ...
166+
end)
167+
168+
{[spec: spec], state}
169+
end
170+
```
171+
172+
The elements that the output pads will be linked to should be based on what stream
173+
format is in `format` variable - different formats require different approaches.
174+
175+
After this spec is returned, the demuxer will now have the
176+
[`handle_pad_added/3`](`c:Membrane.Element.Base.handle_pad_added/3`) callback
177+
called for each new connected pad with pad reference of
178+
`Pad.ref(:output, track_id)`. It will now know that these pads are connected and
179+
ready to pass buffers forward.

0 commit comments

Comments
 (0)