Skip to content

Commit c3aee79

Browse files
committed
Fix snippets for Oban. Move batch processing section down. Add description of terminating pipeline
1 parent f6af6c9 commit c3aee79

1 file changed

Lines changed: 93 additions & 78 deletions

File tree

guides/useful_concepts/running_membrane_in_elixir_application.md

Lines changed: 93 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -6,85 +6,11 @@ This guide outlines best practices for integrating Membrane Pipelines into your
66

77
In most cases, pipelines are used in one of the following scenarios:
88

9-
- Batch Processing: Handling "offline" tasks with a clear start and end (e.g. transcoding a stream inside an MP4 file).
10-
119
- Static Orchestration: Maintaining a single, permanent pipeline that always runs with your application (e.g. mixing output from multiple IP cameras into a single HLS stream).
1210

1311
- Dynamic Orchestration: Spawning pipelines on-demand based on user actions (e.g. starting a unique pipeline for every user joining a conference room).
1412

15-
### Batch Processing
16-
17-
Batch processing is ideal for "offline tasks" where you need to ensure the job completes successfully.
18-
19-
For example, let's assume we want to transcode video from H.264 (in MP4) to VP8 (in Matroska). To achieve this, we build the pipeline as follows:
20-
21-
```elixir
22-
defmodule TranscodePipeline do
23-
use Membrane.Pipeline
24-
25-
alias Membrane.File.{Source, Sink}
26-
alias Membrane.H264.FFmpeg.Parser
27-
alias Membrane.FFmpeg.SWScale.PixelFormatConverter
28-
29-
@impl true
30-
def handle_init(_ctx, opts) do
31-
structure = [
32-
child(:source, %Source{location: opts[:input_path]})
33-
|> child(:demuxer, Membrane.MP4.Demuxer.ISOM)
34-
|> via_out(:video)
35-
|> child(:parser, %Parser{generate_best_effort_timestamps: true})
36-
|> child(:decoder, Membrane.H264.FFmpeg.Decoder)
37-
|> child(:converter, %PixelFormatConverter{format: :I420})
38-
|> child(:encoder, Membrane.VP8.Encoder)
39-
|> child(:muxer, Membrane.Matroska.Muxer)
40-
|> child(:sink, %Sink{location: opts[:output_path]})
41-
]
42-
43-
{[spec: structure], %{}}
44-
end
45-
end
46-
```
47-
48-
To ensure reliability, we might wrap this execution in an [Oban](https://hexdocs.pm/oban/Oban.html) worker.
49-
50-
Oban is the standard library for background job processing in Elixir. It persists jobs to your database, ensuring that your long-running transcoding tasks are fault-tolerant.
51-
If the pipeline crashes or the server restarts, Oban will automatically retry the job until it succeeds.
52-
53-
Let's define an Oban worker:
54-
55-
```elixir
56-
defmodule VideoTranscoderWorker do
57-
@timeout_sec :time
58-
59-
use Oban.Worker, queue: :media, unique: [period: 60], timeout: :timer.seconds(10)
60-
61-
@impl true
62-
def perform(%Oban.Job{args: %{"input_path" => input, "output_path" => output}}) do
63-
{:ok, _supervisor_pid, pipeline_pid} = Membrane.Pipeline.start_link(
64-
TranscodePipeline,
65-
[input_path: input, output_path: output]
66-
)
67-
68-
case Membrane.Pipeline.wait_for_termination(pipeline_pid, 100_000) do
69-
:ok -> :ok
70-
{:error, reason} -> {:error, reason}
71-
end
72-
end
73-
end
74-
```
75-
76-
Now we can run the Oban worker:
77-
78-
```elixir
79-
%{
80-
"input_path" => "uploads/input_video.mp4",
81-
"output_path" => "processed/transcoded_video.mkv"
82-
}
83-
|> VideoTranscoderWorker.new()
84-
|> Oban.insert()
85-
```
86-
87-
When the job terminates successfully, you will see the output `.mkv` file with transcoded video.
13+
- Batch Processing: Handling "offline" tasks with a clear start and end (e.g. transcoding a stream inside an MP4 file).
8814

8915
### Static Orchestration: Supervisor on Startup
9016

@@ -167,17 +93,106 @@ defmodule MyProject.Application do
16793
end
16894
```
16995

170-
Now, whenever a new pipeline is needed you can spawn a new instance of your infrastructure using `DynamicSupervisor.start_child/2`.
96+
Now, whenever a new pipeline is needed you can spawn a new instance of your infrastructure using [`DynamicSupervisor.start_child/2`](https://hexdocs.pm/elixir/1.12/DynamicSupervisor.html#start_child/2).
17197
It could happen e.g. inside `mount/3` callback of your LiveView:
17298

17399
```elixir
174100
def mount(params, _session, socket) do
175101
if connected?(socket) do
176-
{:ok, _pid} = DynamicSupervisor.start_child(
102+
{:ok, infrastructure_supervisor_pid} = DynamicSupervisor.start_child(
177103
MyProject.PipelineDynamicSupervisor,
178104
{MyProject.InfrastructureSupervisor, params["rtsp_url"]}
179105
)
106+
{:ok, assign(socket, :infrastructure_supervisor_pid, infrastructure_supervisor_pid)}
107+
else
108+
{:ok, socket}
180109
end
181-
{:ok, socket}
182110
end
183111
```
112+
113+
When you need to stop the pipeline, you can use [`DynamicSupervisor.terminate_child/2`](https://hexdocs.pm/elixir/1.12/DynamicSupervisor.html#terminate_child/2).
114+
115+
In case of your LiveView commponent it could happen in its `terminate/2` callback:
116+
117+
```elixir
118+
def terminate(_reason, socket) do
119+
if pid = socket.assigns[:infrastructure_supervisor_pid] do
120+
DynamicSupervisor.terminate_child(MyProject.PipelineDynamicSupervisor, pid)
121+
end
122+
:ok
123+
end
124+
```
125+
126+
### Batch Processing
127+
128+
Batch processing is ideal for "offline tasks" where you need to ensure the job completes successfully.
129+
130+
For example, let's assume we want to transcode video from H.264 (in MP4) to VP8 (in Matroska). To achieve this, we build the pipeline as follows:
131+
132+
```elixir
133+
defmodule TranscodePipeline do
134+
use Membrane.Pipeline
135+
136+
alias Membrane.File.{Source, Sink}
137+
alias Membrane.H264.Parser
138+
alias Membrane.FFmpeg.SWScale.PixelFormatConverter
139+
140+
@impl true
141+
def handle_init(_ctx, opts) do
142+
structure = [
143+
child(:source, %Source{location: opts[:input_path]})
144+
|> child(:demuxer, Membrane.MP4.Demuxer.ISOM)
145+
|> via_out(:video)
146+
|> child(:parser, Parser)
147+
|> child(:decoder, Membrane.H264.FFmpeg.Decoder)
148+
|> child(:converter, %PixelFormatConverter{format: :I420})
149+
|> child(:encoder, Membrane.VP8.Encoder)
150+
|> child(:muxer, Membrane.Matroska.Muxer)
151+
|> child(:sink, %Sink{location: opts[:output_path]})
152+
]
153+
154+
{[spec: structure], %{}}
155+
end
156+
end
157+
```
158+
159+
To ensure reliability, we might wrap this execution in an [Oban](https://hexdocs.pm/oban/Oban.html) worker.
160+
161+
Oban is the standard library for background job processing in Elixir. It persists jobs to your database, ensuring that your long-running transcoding tasks are fault-tolerant.
162+
If the pipeline crashes or the server restarts, Oban will automatically retry the job until it succeeds.
163+
164+
Let's define an Oban worker:
165+
166+
```elixir
167+
defmodule VideoTranscoderWorker do
168+
@timeout_sec :time
169+
170+
use Oban.Worker, queue: :media, unique: [period: 60], timeout: :timer.seconds(10)
171+
172+
@impl true
173+
def perform(%Oban.Job{args: %{"input_path" => input, "output_path" => output}}) do
174+
{:ok, _supervisor_pid, pipeline_pid} = Membrane.Pipeline.start_link(
175+
TranscodePipeline,
176+
[input_path: input, output_path: output]
177+
)
178+
179+
case Membrane.Pipeline.wait_for_termination(pipeline_pid, 100_000) do
180+
:ok -> :ok
181+
{:error, reason} -> {:error, reason}
182+
end
183+
end
184+
end
185+
```
186+
187+
Now we can run the Oban worker:
188+
189+
```elixir
190+
%{
191+
"input_path" => "uploads/input_video.mp4",
192+
"output_path" => "processed/transcoded_video.mkv"
193+
}
194+
|> VideoTranscoderWorker.new()
195+
|> Oban.insert()
196+
```
197+
198+
When the job terminates successfully, you will see the output `.mkv` file with transcoded video.

0 commit comments

Comments
 (0)