Skip to content

Commit d892c68

Browse files
committed
Write crash group guide WiP
1 parent 23d947b commit d892c68

1 file changed

Lines changed: 73 additions & 0 deletions

File tree

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Crash groups
2+
3+
Crash groups provide a mechanism to manage the lifecycle of elements within a pipeline when one of them fails. By grouping elements together, you can ensure that a crash in one part of the pipeline triggers a coordinated restart or termination of related elements, maintaining system consistency.
4+
5+
## Overview
6+
7+
In Membrane, elements and bins are Elixir processes. By default, if an element that is not inside a crash group crashes, it leads to the crash of the whole pipeline.
8+
9+
The most fundamental functionality of crash groups is to separate the crash of a specific element from the rest of the pipeline. Usually, if an element is likely to crash (e.g. it interacts with unstable external resources), it is placed in a crash group along with other elements whose functioning is inextricably connected to it. This prevents a localized failure from bringing down the entire system and allows for controlled recovery of specific logical units. By doing so, it also cleans up components that would have to be restarted or killed anyway.
10+
11+
## Defining Crash Groups
12+
13+
Crash groups are defined in the `spec` within your pipeline or bin. You assign a crash group ID to a set of children.
14+
15+
```elixir
16+
defmodule MyPipeline do
17+
use Membrane.Pipeline
18+
19+
@impl true
20+
def handle_init(_ctx, _opts) do
21+
spec =
22+
child(:source, MySource)
23+
|> child(:filter, MyFilter)
24+
|> child(:sink, MySink)
25+
26+
{[spec: {spec, crash_group: :audio_processing}], %{}}
27+
end
28+
end
29+
```
30+
31+
In the case above, the crash group ID is `:audio_processing`.
32+
33+
## Behavior
34+
35+
When an element belonging to a crash group crashes:
36+
1. All other elements in the same crash group are terminated by the pipeline.
37+
2. The pipeline's `handle_crash_group_down/3` callback is invoked.
38+
3. You can decide whether to restart the group, ignore the failure, or terminate the pipeline.
39+
40+
## Handling Failures
41+
42+
To react to a crash group failure, implement the `handle_crash_group_down/3` callback:
43+
44+
```elixir
45+
@impl true
46+
def handle_crash_group_down(crash_group_id, context, state) do
47+
# Logic to restart the group or handle the error
48+
{[], state}
49+
end
50+
```
51+
52+
`context` passed to `handle_crash_group_down/3` callback contiains 3 additional fields, that usually don't occur in contexts of other callbacks:
53+
- `context.crash_initiator` - name or reference of the child that crashed, what caused crash group to explode.
54+
- `context.crash_reason` - the reason with which `context.crash_initiator` crashed.
55+
- `context.members` - names/references of all children that were in the crash group.
56+
57+
Question marks:
58+
- is crash group initiator in context.members?
59+
- what is the order in of:
60+
* `handle_crash_group_down`
61+
* `handle_child_terminated`
62+
* removing children from `context.children`, so that it becames possible to respawn new children with the same names?
63+
- what is the relation between `handle_child_pad_removed` and `handle_crash_group_down`?
64+
65+
66+
67+
## Use Cases
68+
## tutaj poniej mamy AI BS, no chodzi o to ze jak mamy element co sie wydupca, to nie chcemy zeby wszystko poszlo w piach, wiec wrzucamy go (i byc moze cos co i tak bysmy chcieli razem z nim zrestartowac) do crash groupy
69+
70+
- **Atomic logical units:** When a group of elements (like an encoder and its associated parser) cannot function independently.
71+
- **Resource Cleanup:** Ensuring that if a consumer crashes, the producer is also stopped to prevent buffered data from leaking memory.
72+
- **Error Recovery:** Grouping elements that require a specific initialization sequence that must be repeated upon failure.
73+

0 commit comments

Comments
 (0)