Skip to content

Commit 38df1ea

Browse files
FelonEkonomFelonEkonom
committed
Merge remote-tracking branch 'origin/master' into time-on-demands
2 parents b5b96dd + d803d74 commit 38df1ea

6 files changed

Lines changed: 469 additions & 0 deletions

File tree

‎.claude-plugin/marketplace.json‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"name": "membrane-framework",
3+
"description": "Membrane multimedia streaming framework skill for Elixir — provides architectural guidance, callback and action references, and code patterns for building pipelines, elements, and bins with membrane_core.",
4+
"owner": {
5+
"name": "Software Mansion",
6+
"url": "https://github.com/membraneframework"
7+
},
8+
"plugins": [
9+
{
10+
"name": "membrane-core",
11+
"description": "Work with the Membrane multimedia streaming framework in Elixir. Provides architectural guidance, callback/action references, and code patterns for pipelines, elements, and bins.",
12+
"version": "1.0.0",
13+
"source": "./",
14+
"category": "development",
15+
"homepage": "https://github.com/membraneframework/membrane_core",
16+
"skills": [
17+
"./skills/membrane-core"
18+
]
19+
}
20+
]
21+
}

‎.claude/CLAUDE.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Claude Instructions
2+
3+
This is the **membrane_core** repository — the core of the Membrane multimedia streaming framework for Elixir.
4+
5+
Always use the `membrane-core` skill when working in this repository.
6+
7+
## Running Tests
8+
9+
```bash
10+
mix test # run all tests
11+
mix coveralls # with coverage
12+
mix coveralls.html # coverage as HTML report
13+
```
14+
15+
## Project Structure
16+
17+
| Directory | Purpose |
18+
|-----------|---------|
19+
| `lib/` | Main source code |
20+
| `test/` | Tests |
21+
| `test/support/` | Test helpers |
22+
| `guides/` | Documentation and tutorials |
23+
| `benchmark/` | Performance benchmarks |
24+
| `config/` | Configuration |
25+
26+
## Conventions
27+
28+
- Elixir ~> 1.17
29+
- Format code with `mix format`
30+
- Type checking via Dialyzer (`mix dialyzer`)
31+
- Linting via Credo (`mix credo`)
32+
- Never modify code in `deps/`

‎.claude/skills/membrane-core/SKILL.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
name: membrane-core
3+
description: Work with the Membrane multimedia streaming framework in Elixir. Use this skill whenever the user is building or debugging Membrane pipelines, writing custom Elements, Bins, or Filters, connecting pads, implementing callbacks, handling stream formats or EOS, or asking about Membrane architecture. Trigger on any mention of membrane_core, Membrane.Pipeline, Membrane.Element, Membrane.Bin, Membrane.Pad, or multimedia streaming in an Elixir context — even if the user doesn't say "Membrane" explicitly but is clearly working on this codebase.
4+
---
5+
6+
# Membrane Framework
7+
8+
**Package**: `membrane_core` ~> 1.2 | **Docs**: https://hexdocs.pm/membrane_core/ | **Demos**: https://github.com/membraneframework/membrane_demo
9+
10+
## How to Approach Tasks
11+
12+
- **New component** — identify subtype (Source/Filter/Sink/Endpoint/Bin), define pads, implement required callbacks (`handle_buffer/4` for filters/sinks, `handle_demand/5` for manual-flow sources)
13+
- **Pipeline topology** — use the ChildrenSpec DSL (`child/2`, `get_child/1`, `via_in/2`, `via_out/2`); see Integration Patterns below or `references/integration.md`
14+
- **Dynamic tracks** (demuxers, variable inputs) — use the Dynamic Pads Pattern below
15+
- **Debugging** — check pad `accepted_format` compatibility, lifecycle ordering (`handle_setup` vs `handle_playing`), flow control mode mismatches
16+
- **Full callback reference** — `references/callbacks.md`
17+
- **Full actions reference** — `references/actions.md`
18+
- **Never modify code in `deps/`**
19+
- **Use `mix hex.info <plugin name>` when you need to check the newest version of a plugin**
20+
- **Search for appropriate plugins in `README.md`, in `all-packages` section**
21+
- **Check input and output pad definitions of elements in `deps/` (use `cat <filename> | grep def_input_pad` and `cat <filename> | grep def_output pad`) to make sure output pad's `accepted_stream_format` is compatible with `accepted_stream_format` of the input pad which it is linked to.**
22+
- **If the `accepted_stream_format` doesn't match, search for an element which can act as an adapter**
23+
24+
---
25+
26+
## Architecture
27+
28+
```
29+
Pipeline
30+
├── Element (Source/Filter/Sink/Endpoint) ← leaf, processes data
31+
├── Bin ← dual role: parent (has children) + child (has pads)
32+
│ ├── Element
33+
│ └── Bin ← bins nest arbitrarily deep
34+
└── ...
35+
```
36+
37+
| Type | Parent | Child | Has Pads |
38+
|------|--------|-------|----------|
39+
| **Pipeline** | yes | no | no |
40+
| **Bin** | yes | yes | yes |
41+
| **Element** | no | yes | yes |
42+
43+
Element subtypes: **Source** (output only) · **Filter** (in + out, output is transformed input) · **Sink** (input only) · **Endpoint** (in + out, but output might be not related to input)
44+
45+
---
46+
47+
## Pads
48+
49+
Defined on Elements and Bins (not Pipelines):
50+
51+
```elixir
52+
def_input_pad :input, accepted_format: _any
53+
def_output_pad :output, accepted_format: Membrane.RawAudio, flow_control: :auto
54+
```
55+
56+
- **Availability**: `:always` (static, one instance, referenced by atom) or `:on_request` (dynamic, reference via `Pad.ref(:name, id)`)
57+
- **Flow control**: `:auto` (framework manages demand — preferred), `:manual` (explicit via `:demand`/`:redemand`), `:push` (no demand, risk of overflow)
58+
- One input pad ↔ one output pad only; pads must have compatible `accepted_format`
59+
- Default pad names `:input`/`:output` allow omitting `via_in`/`via_out` in specs
60+
61+
---
62+
63+
## Component Lifecycle
64+
65+
```
66+
handle_init/2 sync, blocks parent — parse opts, return initial spec
67+
handle_setup/2 async — heavy init (open files, connect services)
68+
return {[setup: :incomplete], state} to delay :playing
69+
handle_pad_added/3 fires for dynamic pads linked in the same spec
70+
handle_playing/2 component is ready — start producing/consuming data
71+
```
72+
73+
All components spawned in the same `:spec` action enter `:playing` together (slowest setup wins). Elements and Bins wait for their parent before `handle_playing/2`.
74+
75+
---
76+
77+
## Pipeline & Bin DSL (ChildrenSpec)
78+
79+
```elixir
80+
# Linear chain — child/2 spawns a new named child
81+
child(:source, %Membrane.File.Source{location: "input.mp4"})
82+
|> child(:filter, MyFilter)
83+
|> child(:sink, %Membrane.File.Sink{location: "out.raw"})
84+
85+
# Explicit pad names (required for non-default names or dynamic pads)
86+
get_child(:demuxer)
87+
|> via_out(Pad.ref(:output, track_id))
88+
|> via_in(:video_input)
89+
|> child(:decoder, Membrane.H264.FFmpeg.Decoder)
90+
91+
# Link to an already-existing child
92+
get_child(:existing_filter) |> child(:new_sink, MySink)
93+
94+
# Inside a Bin
95+
bin_input(:input) |> child(:filter, MyFilter) |> bin_output(:output)
96+
```
97+
98+
---
99+
100+
## Dynamic Pads Pattern
101+
102+
The standard approach for variable-track streams (e.g. MP4 demuxers):
103+
104+
```elixir
105+
# 1. Spawn source + demuxer; demuxer hasn't identified tracks yet
106+
def handle_init(_ctx, state) do
107+
{[spec: child(:source, Source) |> child(:demuxer, Demuxer)], state}
108+
end
109+
110+
# 2. Demuxer notifies parent once tracks are known
111+
def handle_child_notification({:new_tracks, tracks}, :demuxer, _ctx, state) do
112+
spec = Enum.map(tracks, fn {id, _fmt} ->
113+
get_child(:demuxer)
114+
|> via_out(Pad.ref(:output, id))
115+
|> child({:decoder, id}, Decoder)
116+
|> child({:sink, id}, Sink)
117+
end)
118+
{[spec: spec], state}
119+
end
120+
```
121+
122+
---
123+
124+
## Built-in Utility Elements
125+
126+
| Module | Purpose |
127+
|--------|---------|
128+
| `Membrane.Funnel` | Multiple inputs → one output |
129+
| `Membrane.Tee` | One input → multiple outputs |
130+
| `Membrane.Connector` | Connect dynamic pads with internal buffering |
131+
| `Membrane.FilterAggregator` | Run multiple filters in a single process |
132+
| `Membrane.Testing.Source` | Inject buffers into a pipeline in tests |
133+
| `Membrane.Testing.Sink` | Capture and assert on buffers in tests |
134+
135+
---
136+
137+
## Testing
138+
139+
```elixir
140+
import Membrane.ChildrenSpec
141+
import Membrane.Testing.Assertions
142+
alias Membrane.Testing
143+
144+
pipeline = Testing.Pipeline.start_link_supervised!(spec: [
145+
child(:source, %Testing.Source{output: [<<1, 2, 3>>, <<4, 5, 6>>]})
146+
|> child(:sink, Testing.Sink)
147+
])
148+
149+
assert_sink_buffer(pipeline, :sink, %Membrane.Buffer{payload: <<1, 2, 3>>})
150+
```
151+
152+
---
153+
154+
## Timing
155+
156+
All timestamps are `Membrane.Time.t()` (integer nanoseconds). Helpers: `Membrane.Time.seconds/1`, `Membrane.Time.milliseconds/1`, `Membrane.Time.microseconds/1`, etc. Timers started with `:start_timer` action fire `handle_tick/3`.
157+
158+
---
159+
160+
## Reference Files
161+
162+
- `references/callbacks.md` — full callback tables for every component type
163+
- `references/actions.md` — all actions with signatures and usage notes
164+
- `references/integration.md` — integration patterns, demo examples, key source file locations

‎.claude/skills/membrane-core/references/actions.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Membrane Framework — Actions Reference
2+
3+
Actions are returned from callbacks as `{[action_list], state}`.
4+
5+
---
6+
7+
## Data Actions (Elements Only)
8+
9+
```elixir
10+
{:buffer, {pad_ref, %Membrane.Buffer{payload: binary, pts: pts, dts: dts, metadata: map}}}
11+
# Send a buffer on an output pad
12+
13+
{:stream_format, {pad_ref, %SomeFormat{}}}
14+
# Send stream format on an output pad — must be sent before the first buffer
15+
16+
{:event, {pad_ref, %SomeEvent{}}}
17+
# Send an event on any pad — can go upstream or downstream
18+
19+
{:end_of_stream, pad_ref}
20+
# Signal end of stream on an output pad
21+
```
22+
23+
---
24+
25+
## Flow Control Actions
26+
27+
```elixir
28+
# Manual input pads (flow_control: :manual) only:
29+
{:demand, {pad_ref, size}} # request `size` more units from upstream
30+
{:redemand, pad_ref} # re-evaluate demand on an output pad
31+
32+
# Auto input pads (flow_control: :auto) only:
33+
{:pause_auto_demand, pad_ref} # temporarily stop auto-pulling from upstream
34+
{:resume_auto_demand, pad_ref} # resume auto-pulling
35+
```
36+
37+
---
38+
39+
## Topology Actions (Pipeline and Bin)
40+
41+
```elixir
42+
{:spec, children_spec} # spawn and/or link children
43+
{:remove_children, name_or_list} # stop and remove children by name
44+
{:remove_link, {child_name, pad_ref}} # unlink a dynamic pad only
45+
```
46+
47+
---
48+
49+
## Communication Actions
50+
51+
```elixir
52+
{:notify_parent, any_term} # send notification to parent (Bin and Elements only)
53+
# received in parent's handle_child_notification/4
54+
```
55+
56+
---
57+
58+
## Component Control Actions
59+
60+
```elixir
61+
{:setup, :complete | :incomplete}
62+
# :incomplete — signals setup isn't done yet, delays :playing for the whole spec group
63+
# :complete — signals setup is done (use after returning :incomplete earlier)
64+
65+
{:terminate, reason} # stop this component with the given reason
66+
```
67+
68+
---
69+
70+
## Timer Actions
71+
72+
```elixir
73+
{:start_timer, {name, interval}} # start a periodic timer → fires handle_tick/3
74+
{:stop_timer, name} # stop a named timer
75+
{:timer_interval, {name, new_interval}} # change interval of a running timer
76+
```
77+
78+
`interval` is a `Membrane.Time.t()` value (integer nanoseconds). Use helpers like `Membrane.Time.milliseconds(100)`.
79+
80+
---
81+
82+
## Buffer Struct Reference
83+
84+
```elixir
85+
%Membrane.Buffer{
86+
payload: binary, # actual media data
87+
pts: Membrane.Time.t() | nil, # presentation timestamp (nanoseconds)
88+
dts: Membrane.Time.t() | nil, # decode timestamp (nanoseconds)
89+
metadata: map | nil # arbitrary extra info
90+
}
91+
```

‎.claude/skills/membrane-core/references/callbacks.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Membrane Framework — Callback Reference
2+
3+
## Every Component (Pipeline, Bin, Element)
4+
5+
| Callback | When called | Default |
6+
|----------|-------------|---------|
7+
| `handle_init/2` | Spawned (sync, blocks parent) | converts opts struct to map |
8+
| `handle_setup/2` | After init (async) | no-op |
9+
| `handle_playing/2` | Entering `:playing` | no-op |
10+
| `handle_terminate_request/2` | Termination requested | `{[terminate: :normal], state}` |
11+
| `handle_info/3` | Any non-Membrane Erlang message | logs warning |
12+
| `handle_tick/3` | Timer tick (started with `:start_timer` action) | no-op |
13+
14+
---
15+
16+
## Parent Components Only (Pipeline and Bin)
17+
18+
| Callback | When called | Default |
19+
|----------|-------------|---------|
20+
| `handle_child_notification/4` | Child called `notify_parent` | no-op |
21+
| `handle_child_setup_completed/3` | Child finished `handle_setup` | no-op |
22+
| `handle_child_playing/3` | Child entered `:playing` | no-op |
23+
| `handle_child_pad_removed/4` | Child removed a pad (child-initiated only) | no-op |
24+
| `handle_child_terminated/3` | A child process terminated | no-op |
25+
| `handle_crash_group_down/3` | All children in a crash group are down | no-op |
26+
| `handle_element_start_of_stream/4` | A child element's pad started streaming | no-op |
27+
| `handle_element_end_of_stream/4` | EOS received on a child element's input pad **or** sent on its output pad | no-op |
28+
29+
---
30+
31+
## Child Components Only (Bin and all Elements)
32+
33+
| Callback | When called | Default |
34+
|----------|-------------|---------|
35+
| `handle_parent_notification/3` | Parent sent a notification | no-op |
36+
| `handle_pad_added/3` | Dynamic pad linked | no-op |
37+
| `handle_pad_removed/3` | Dynamic pad unlinked | no-op |
38+
39+
**Lifecycle note for `handle_pad_added/3`:**
40+
- Fires *before* `handle_playing/2` when the dynamic pad is linked in **the same spec** that spawns the component
41+
- Fires *after* `handle_playing/2` when the pad is linked in a **later spec**
42+
43+
---
44+
45+
## Pipeline Only
46+
47+
| Callback | When called | Default |
48+
|----------|-------------|---------|
49+
| `handle_call/3` | Synchronous call from an external process | no-op |
50+
51+
---
52+
53+
## Elements with Input Pads (Filter, Sink, Endpoint)
54+
55+
| Callback | When called | Default |
56+
|----------|-------------|---------|
57+
| `handle_stream_format/4` | Stream format received on input pad | Filter: forwards downstream |
58+
| `handle_start_of_stream/3` | First buffer about to arrive on input pad | no-op |
59+
| `handle_buffer/4` | Buffer received on input pad | — (no default, must implement) |
60+
| `handle_end_of_stream/3` | EOS received on element's own input pad | Filter: forwards downstream |
61+
62+
---
63+
64+
## Elements with Output Pads (Source, Filter, Endpoint)
65+
66+
| Callback | When called | Default |
67+
|----------|-------------|---------|
68+
| `handle_demand/5` | Downstream requested data on a `:manual` output pad | — (no default, must implement if using `:manual` flow) |
69+
70+
---
71+
72+
## Events (All Elements)
73+
74+
| Callback | When called | Default |
75+
|----------|-------------|---------|
76+
| `handle_event/4` | Event received on any pad | Filter: forwards; others: no-op |
77+
78+
**Key property of events**: unlike buffers and stream formats, events can travel both **upstream and downstream**.

0 commit comments

Comments
 (0)