diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 56dd3f8fd..20ae6af40 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -14,7 +14,7 @@ "category": "development", "homepage": "https://github.com/membraneframework/membrane_core", "skills": [ - "./skills/membrane-core" + "./skills/membrane-framework" ] } ] diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md deleted file mode 100644 index 4ba99f1af..000000000 --- a/.claude/CLAUDE.md +++ /dev/null @@ -1,32 +0,0 @@ -# Claude Instructions - -This is the **membrane_core** repository — the core of the Membrane multimedia streaming framework for Elixir. - -Always use the `membrane-core` skill when working in this repository. - -## Running Tests - -```bash -mix test # run all tests -mix coveralls # with coverage -mix coveralls.html # coverage as HTML report -``` - -## Project Structure - -| Directory | Purpose | -|-----------|---------| -| `lib/` | Main source code | -| `test/` | Tests | -| `test/support/` | Test helpers | -| `guides/` | Documentation and tutorials | -| `benchmark/` | Performance benchmarks | -| `config/` | Configuration | - -## Conventions - -- Elixir ~> 1.17 -- Format code with `mix format` -- Type checking via Dialyzer (`mix dialyzer`) -- Linting via Credo (`mix credo`) -- Never modify code in `deps/` diff --git a/.claude/skills/membrane-core/SKILL.md b/.claude/skills/membrane-core/SKILL.md deleted file mode 100644 index 1aa80c90b..000000000 --- a/.claude/skills/membrane-core/SKILL.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -name: membrane-core -description: Work with the Membrane multimedia streaming framework in Elixir. Use this skill whenever the user is building or debugging Membrane pipelines, writing custom Elements, Bins, or Filters, connecting pads, implementing callbacks, handling stream formats or EOS, or asking about Membrane architecture. Trigger on any mention of membrane_core, Membrane.Pipeline, Membrane.Element, Membrane.Bin, Membrane.Pad, or multimedia streaming in an Elixir context — even if the user doesn't say "Membrane" explicitly but is clearly working on this codebase. ---- - -# Membrane Framework - -**Package**: `membrane_core` ~> 1.2 | **Docs**: https://hexdocs.pm/membrane_core/ | **Demos**: https://github.com/membraneframework/membrane_demo - -## How to Approach Tasks - -- **New component** — identify subtype (Source/Filter/Sink/Endpoint/Bin), define pads, implement required callbacks (`handle_buffer/4` for filters/sinks, `handle_demand/5` for manual-flow sources) -- **Pipeline topology** — use the ChildrenSpec DSL (`child/2`, `get_child/1`, `via_in/2`, `via_out/2`); see Integration Patterns below or `references/integration.md` -- **Dynamic tracks** (demuxers, variable inputs) — use the Dynamic Pads Pattern below -- **Debugging** — check pad `accepted_format` compatibility, lifecycle ordering (`handle_setup` vs `handle_playing`), flow control mode mismatches -- **Full callback reference** — `references/callbacks.md` -- **Full actions reference** — `references/actions.md` -- **Never modify code in `deps/`** -- **Use `mix hex.info ` when you need to check the newest version of a plugin** -- **Search for appropriate plugins in `README.md`, in `all-packages` section** -- **Check input and output pad definitions of elements in `deps/` (use `cat | grep def_input_pad` and `cat | grep def_output pad`) to make sure output pad's `accepted_stream_format` is compatible with `accepted_stream_format` of the input pad which it is linked to.** -- **If the `accepted_stream_format` doesn't match, search for an element which can act as an adapter** - ---- - -## Architecture - -``` -Pipeline -├── Element (Source/Filter/Sink/Endpoint) ← leaf, processes data -├── Bin ← dual role: parent (has children) + child (has pads) -│ ├── Element -│ └── Bin ← bins nest arbitrarily deep -└── ... -``` - -| Type | Parent | Child | Has Pads | -|------|--------|-------|----------| -| **Pipeline** | yes | no | no | -| **Bin** | yes | yes | yes | -| **Element** | no | yes | yes | - -Element subtypes: **Source** (output only) · **Filter** (in + out, output is transformed input) · **Sink** (input only) · **Endpoint** (in + out, but output might be not related to input) - ---- - -## Pads - -Defined on Elements and Bins (not Pipelines): - -```elixir -def_input_pad :input, accepted_format: _any -def_output_pad :output, accepted_format: Membrane.RawAudio, flow_control: :auto -``` - -- **Availability**: `:always` (static, one instance, referenced by atom) or `:on_request` (dynamic, reference via `Pad.ref(:name, id)`) -- **Flow control**: `:auto` (framework manages demand — preferred), `:manual` (explicit via `:demand`/`:redemand`), `:push` (no demand, risk of overflow) -- One input pad ↔ one output pad only; pads must have compatible `accepted_format` -- Default pad names `:input`/`:output` allow omitting `via_in`/`via_out` in specs - ---- - -## Component Lifecycle - -``` -handle_init/2 sync, blocks parent — parse opts, return initial spec -handle_setup/2 async — heavy init (open files, connect services) - return {[setup: :incomplete], state} to delay :playing -handle_pad_added/3 fires for dynamic pads linked in the same spec -handle_playing/2 component is ready — start producing/consuming data -``` - -All components spawned in the same `:spec` action enter `:playing` together (slowest setup wins). Elements and Bins wait for their parent before `handle_playing/2`. - ---- - -## Pipeline & Bin DSL (ChildrenSpec) - -```elixir -# Linear chain — child/2 spawns a new named child -child(:source, %Membrane.File.Source{location: "input.mp4"}) -|> child(:filter, MyFilter) -|> child(:sink, %Membrane.File.Sink{location: "out.raw"}) - -# Explicit pad names (required for non-default names or dynamic pads) -get_child(:demuxer) -|> via_out(Pad.ref(:output, track_id)) -|> via_in(:video_input) -|> child(:decoder, Membrane.H264.FFmpeg.Decoder) - -# Link to an already-existing child -get_child(:existing_filter) |> child(:new_sink, MySink) - -# Inside a Bin -bin_input(:input) |> child(:filter, MyFilter) |> bin_output(:output) -``` - ---- - -## Dynamic Pads Pattern - -The standard approach for variable-track streams (e.g. MP4 demuxers): - -```elixir -# 1. Spawn source + demuxer; demuxer hasn't identified tracks yet -def handle_init(_ctx, state) do - {[spec: child(:source, Source) |> child(:demuxer, Demuxer)], state} -end - -# 2. Demuxer notifies parent once tracks are known -def handle_child_notification({:new_tracks, tracks}, :demuxer, _ctx, state) do - spec = Enum.map(tracks, fn {id, _fmt} -> - get_child(:demuxer) - |> via_out(Pad.ref(:output, id)) - |> child({:decoder, id}, Decoder) - |> child({:sink, id}, Sink) - end) - {[spec: spec], state} -end -``` - ---- - -## Built-in Utility Elements - -| Module | Purpose | -|--------|---------| -| `Membrane.Funnel` | Multiple inputs → one output | -| `Membrane.Tee` | One input → multiple outputs | -| `Membrane.Connector` | Connect dynamic pads with internal buffering | -| `Membrane.FilterAggregator` | Run multiple filters in a single process | -| `Membrane.Testing.Source` | Inject buffers into a pipeline in tests | -| `Membrane.Testing.Sink` | Capture and assert on buffers in tests | - ---- - -## Testing - -```elixir -import Membrane.ChildrenSpec -import Membrane.Testing.Assertions -alias Membrane.Testing - -pipeline = Testing.Pipeline.start_link_supervised!(spec: [ - child(:source, %Testing.Source{output: [<<1, 2, 3>>, <<4, 5, 6>>]}) - |> child(:sink, Testing.Sink) -]) - -assert_sink_buffer(pipeline, :sink, %Membrane.Buffer{payload: <<1, 2, 3>>}) -``` - ---- - -## Timing - -All timestamps are `Membrane.Time.t()` (integer nanoseconds). Helpers: `Membrane.Time.seconds/1`, `Membrane.Time.milliseconds/1`, `Membrane.Time.microseconds/1`, etc. Timers started with `:start_timer` action fire `handle_tick/3`. - ---- - -## Reference Files - -- `references/callbacks.md` — full callback tables for every component type -- `references/actions.md` — all actions with signatures and usage notes -- `references/integration.md` — integration patterns, demo examples, key source file locations diff --git a/.claude/skills/membrane-core/references/actions.md b/.claude/skills/membrane-core/references/actions.md deleted file mode 100644 index 9c7b30220..000000000 --- a/.claude/skills/membrane-core/references/actions.md +++ /dev/null @@ -1,91 +0,0 @@ -# Membrane Framework — Actions Reference - -Actions are returned from callbacks as `{[action_list], state}`. - ---- - -## Data Actions (Elements Only) - -```elixir -{:buffer, {pad_ref, %Membrane.Buffer{payload: binary, pts: pts, dts: dts, metadata: map}}} -# Send a buffer on an output pad - -{:stream_format, {pad_ref, %SomeFormat{}}} -# Send stream format on an output pad — must be sent before the first buffer - -{:event, {pad_ref, %SomeEvent{}}} -# Send an event on any pad — can go upstream or downstream - -{:end_of_stream, pad_ref} -# Signal end of stream on an output pad -``` - ---- - -## Flow Control Actions - -```elixir -# Manual input pads (flow_control: :manual) only: -{:demand, {pad_ref, size}} # request `size` more units from upstream -{:redemand, pad_ref} # re-evaluate demand on an output pad - -# Auto input pads (flow_control: :auto) only: -{:pause_auto_demand, pad_ref} # temporarily stop auto-pulling from upstream -{:resume_auto_demand, pad_ref} # resume auto-pulling -``` - ---- - -## Topology Actions (Pipeline and Bin) - -```elixir -{:spec, children_spec} # spawn and/or link children -{:remove_children, name_or_list} # stop and remove children by name -{:remove_link, {child_name, pad_ref}} # unlink a dynamic pad only -``` - ---- - -## Communication Actions - -```elixir -{:notify_parent, any_term} # send notification to parent (Bin and Elements only) - # received in parent's handle_child_notification/4 -``` - ---- - -## Component Control Actions - -```elixir -{:setup, :complete | :incomplete} -# :incomplete — signals setup isn't done yet, delays :playing for the whole spec group -# :complete — signals setup is done (use after returning :incomplete earlier) - -{:terminate, reason} # stop this component with the given reason -``` - ---- - -## Timer Actions - -```elixir -{:start_timer, {name, interval}} # start a periodic timer → fires handle_tick/3 -{:stop_timer, name} # stop a named timer -{:timer_interval, {name, new_interval}} # change interval of a running timer -``` - -`interval` is a `Membrane.Time.t()` value (integer nanoseconds). Use helpers like `Membrane.Time.milliseconds(100)`. - ---- - -## Buffer Struct Reference - -```elixir -%Membrane.Buffer{ - payload: binary, # actual media data - pts: Membrane.Time.t() | nil, # presentation timestamp (nanoseconds) - dts: Membrane.Time.t() | nil, # decode timestamp (nanoseconds) - metadata: map | nil # arbitrary extra info -} -``` diff --git a/.claude/skills/membrane-core/references/callbacks.md b/.claude/skills/membrane-core/references/callbacks.md deleted file mode 100644 index 06ec3630a..000000000 --- a/.claude/skills/membrane-core/references/callbacks.md +++ /dev/null @@ -1,78 +0,0 @@ -# Membrane Framework — Callback Reference - -## Every Component (Pipeline, Bin, Element) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_init/2` | Spawned (sync, blocks parent) | converts opts struct to map | -| `handle_setup/2` | After init (async) | no-op | -| `handle_playing/2` | Entering `:playing` | no-op | -| `handle_terminate_request/2` | Termination requested | `{[terminate: :normal], state}` | -| `handle_info/3` | Any non-Membrane Erlang message | logs warning | -| `handle_tick/3` | Timer tick (started with `:start_timer` action) | no-op | - ---- - -## Parent Components Only (Pipeline and Bin) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_child_notification/4` | Child called `notify_parent` | no-op | -| `handle_child_setup_completed/3` | Child finished `handle_setup` | no-op | -| `handle_child_playing/3` | Child entered `:playing` | no-op | -| `handle_child_pad_removed/4` | Child removed a pad (child-initiated only) | no-op | -| `handle_child_terminated/3` | A child process terminated | no-op | -| `handle_crash_group_down/3` | All children in a crash group are down | no-op | -| `handle_element_start_of_stream/4` | A child element's pad started streaming | no-op | -| `handle_element_end_of_stream/4` | EOS received on a child element's input pad **or** sent on its output pad | no-op | - ---- - -## Child Components Only (Bin and all Elements) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_parent_notification/3` | Parent sent a notification | no-op | -| `handle_pad_added/3` | Dynamic pad linked | no-op | -| `handle_pad_removed/3` | Dynamic pad unlinked | no-op | - -**Lifecycle note for `handle_pad_added/3`:** -- Fires *before* `handle_playing/2` when the dynamic pad is linked in **the same spec** that spawns the component -- Fires *after* `handle_playing/2` when the pad is linked in a **later spec** - ---- - -## Pipeline Only - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_call/3` | Synchronous call from an external process | no-op | - ---- - -## Elements with Input Pads (Filter, Sink, Endpoint) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_stream_format/4` | Stream format received on input pad | Filter: forwards downstream | -| `handle_start_of_stream/3` | First buffer about to arrive on input pad | no-op | -| `handle_buffer/4` | Buffer received on input pad | — (no default, must implement) | -| `handle_end_of_stream/3` | EOS received on element's own input pad | Filter: forwards downstream | - ---- - -## Elements with Output Pads (Source, Filter, Endpoint) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_demand/5` | Downstream requested data on a `:manual` output pad | — (no default, must implement if using `:manual` flow) | - ---- - -## Events (All Elements) - -| Callback | When called | Default | -|----------|-------------|---------| -| `handle_event/4` | Event received on any pad | Filter: forwards; others: no-op | - -**Key property of events**: unlike buffers and stream formats, events can travel both **upstream and downstream**. diff --git a/.claude/skills/membrane-core/references/integration.md b/.claude/skills/membrane-core/references/integration.md deleted file mode 100644 index 264210f1b..000000000 --- a/.claude/skills/membrane-core/references/integration.md +++ /dev/null @@ -1,83 +0,0 @@ -# Membrane Framework — Integration & Examples Reference - -## Integration Patterns - -### Static — Always-on Pipeline (Application Supervisor) - -```elixir -# application.ex -children = [ - %{ - id: Pipeline, - start: {Membrane.Pipeline, :start_link, [MyPipeline, init_args]}, - restart: :transient - } -] -Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor) -``` - -### Dynamic — Per-User / Per-Request Pipelines - -```elixir -# application.ex — add a DynamicSupervisor -{DynamicSupervisor, strategy: :one_for_one, name: MyApp.PipelineSupervisor} - -# Spawn on demand (e.g. in a LiveView mount or HTTP handler) -{:ok, _sup} = DynamicSupervisor.start_child( - MyApp.PipelineSupervisor, - {Membrane.Pipeline, [MyPipeline, opts]} -) - -# Tear down -DynamicSupervisor.terminate_child(MyApp.PipelineSupervisor, sup_pid) -``` - -### Batch — Wait for Pipeline to Finish - -```elixir -{:ok, _sup, pipeline_pid} = Membrane.Pipeline.start_link(TranscodePipeline, opts) -ref = Process.monitor(pipeline_pid) -receive do - {:DOWN, ^ref, :process, ^pipeline_pid, :normal} -> :ok -end -``` - ---- - -## Demo Examples - -The [`membraneframework/membrane_demo`](https://github.com/membraneframework/membrane_demo) monorepo has runnable examples: - -| Directory | What it demonstrates | -|-----------|---------------------| -| `simple_pipeline` | Minimal pipeline: HTTP → MP3 decode → speaker | -| `simple_element` | Writing a custom element (buffer counter) | -| `camera_to_hls` | Capture webcam and broadcast via HLS | -| `rtmp_to_hls` | Ingest RTMP, broadcast HLS | -| `rtmp_to_adaptive_hls` | Multi-bitrate adaptive HLS from RTMP | -| `rtsp_to_hls` | RTSP stream → HLS | -| `rtp` | Sending and receiving RTP/SRTP streams | -| `rtp_to_hls` | RTP input → HLS output | -| `webrtc_live_view` | WebRTC streaming with Phoenix LiveView | -| `mix_audio` | Audio mixing | - -Livebook notebooks: `playing_mp3_file`, `audio_mixer`, `messages_source_and_sink`, `speech_to_text`, `openai_realtime_with_membrane_webrtc`. - ---- - -## Key Source File Locations - -| File | Purpose | -|------|---------| -| `lib/membrane/pipeline.ex` | Pipeline behaviour & all callbacks | -| `lib/membrane/bin.ex` | Bin behaviour & all callbacks | -| `lib/membrane/element/base.ex` | Shared element callbacks | -| `lib/membrane/element/with_input_pads.ex` | Input-side element callbacks | -| `lib/membrane/element/with_output_pads.ex` | `handle_demand/5` | -| `lib/membrane/pad.ex` | Pad definitions, `Pad.ref/2` | -| `lib/membrane/buffer.ex` | Buffer struct | -| `lib/membrane/children_spec.ex` | Topology DSL | -| `lib/membrane/element/action.ex` | Action type specs | -| `guides/useful_concepts/pads.md` | Pads deep-dive with examples | -| `guides/useful_concepts/components_lifecycle.md` | Lifecycle guide | -| `guides/useful_concepts/running_membrane_in_elixir_application.md` | Integration patterns | diff --git a/README.md b/README.md index d069ba070..ed2ed1f88 100644 --- a/README.md +++ b/README.md @@ -144,8 +144,8 @@ If you have any questions regarding Membrane Framework or need consulting, feel -### General +### General | Package | Description | Links | | --- | --- | --- | | [membrane_sdk](https://github.com/membraneframework/membrane_sdk) | Full power of Membrane in a single package | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_sdk.svg)](https://hex.pm/api/packages/membrane_sdk) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_sdk/) | @@ -160,7 +160,6 @@ If you have any questions regarding Membrane Framework or need consulting, feel ### Plugins #### General purpose - | Package | Description | Links | | --- | --- | --- | | [membrane_file_plugin](https://github.com/membraneframework/membrane_file_plugin) | Membrane plugin for reading and writing to files | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_file_plugin.svg)](https://hex.pm/api/packages/membrane_file_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_file_plugin/) | @@ -174,11 +173,16 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_pcap_plugin](https://github.com/membraneframework-labs/membrane_pcap_plugin) | [Labs] Membrane PCAP source, capable of reading captured packets in pcap format | | | [membrane_transcoder_plugin](https://github.com/membraneframework/membrane_transcoder_plugin) | Membrane plugin providing audio and video transcoding capabilities | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_transcoder_plugin.svg)](https://hex.pm/api/packages/membrane_transcoder_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_transcoder_plugin/) | | [membrane_generator_plugin](https://github.com/membraneframework/membrane_generator_plugin) | Video and audio samples generator | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_generator_plugin.svg)](https://hex.pm/api/packages/membrane_generator_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_generator_plugin/) | -| [membrane_live_framerate_converter_plugin](https://github.com/kim-company/membrane_live_framerate_converter_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that drops or duplicates frames to match a target framerate. Designed for realtime applications | | +| [membrane_live_framerate_converter_plugin](https://github.com/kim-company/membrane_live_framerate_converter_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that drops or duplicates frames to match a target framerate. Designed for realtime applications | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_live_framerate_converter_plugin.svg)](https://hex.pm/api/packages/membrane_live_framerate_converter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_live_framerate_converter_plugin/) | | [membrane_template_plugin](https://github.com/membraneframework/membrane_template_plugin) | Template for Membrane Elements | | -#### Streaming protocols +#### AI +| Package | Description | Links | +| --- | --- | --- | +| [membrane_whisper_plugin](https://github.com/membraneframework/membrane_whisper_plugin) | Membrane plugin for integrating OpenAI's Whisper in audio processing pipelines | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_whisper_plugin.svg)](https://hex.pm/api/packages/membrane_whisper_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_whisper_plugin/) | +| [membrane_yolo_plugin](https://github.com/membraneframework/membrane_yolo_plugin) | Membrane Plugin for applying YOLO object detection on raw video frames | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_yolo_plugin.svg)](https://hex.pm/api/packages/membrane_yolo_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_yolo_plugin/) | +#### Streaming protocols | Package | Description | Links | | --- | --- | --- | | [membrane_webrtc_plugin](https://github.com/membraneframework/membrane_webrtc_plugin) | Plugin for streaming via WebRTC | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_webrtc_plugin.svg)](https://hex.pm/api/packages/membrane_webrtc_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_webrtc_plugin/) | @@ -200,7 +204,6 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_hls_plugin](https://github.com/kim-company/membrane_hls_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Adaptive live streaming plugin (HLS) for the Membrane Framework | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_hls_plugin.svg)](https://hex.pm/api/packages/membrane_hls_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_hls_plugin/) | #### Containers - | Package | Description | Links | | --- | --- | --- | | [membrane_mp4_plugin](https://github.com/membraneframework/membrane_mp4_plugin) | Utilities for MP4 container parsing and serialization and elements for muxing the stream to CMAF | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mp4_plugin.svg)](https://hex.pm/api/packages/membrane_mp4_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mp4_plugin/) | @@ -210,7 +213,6 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_ogg_plugin](https://github.com/membraneframework/membrane_ogg_plugin) | Plugin for depayloading an Ogg file into an Opus stream | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ogg_plugin.svg)](https://hex.pm/api/packages/membrane_ogg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ogg_plugin/) | #### Audio codecs - | Package | Description | Links | | --- | --- | --- | | [membrane_aac_plugin](https://github.com/membraneframework/membrane_aac_plugin) | AAC parser and complementary elements for AAC codec | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_aac_plugin.svg)](https://hex.pm/api/packages/membrane_aac_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_aac_plugin/) | @@ -224,19 +226,18 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_g711_ffmpeg_plugin](https://github.com/membraneframework/membrane_g711_ffmpeg_plugin) | Membrane G.711 decoder and encoder based on FFmpeg | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_g711_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_g711_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_g711_ffmpeg_plugin/) | #### Video codecs - | Package | Description | Links | | --- | --- | --- | | [membrane_h26x_plugin](https://github.com/membraneframework/membrane_h26x_plugin) | Membrane h264 and h265 parsers | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h26x_plugin.svg)](https://hex.pm/api/packages/membrane_h26x_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h26x_plugin/) | | [membrane_h264_ffmpeg_plugin](https://github.com/membraneframework/membrane_h264_ffmpeg_plugin) | Membrane H264 decoder and encoder based on FFmpeg and x264 | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h264_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_h264_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h264_ffmpeg_plugin/) | | [membrane_vpx_plugin](https://github.com/membraneframework/membrane_vpx_plugin) | Membrane plugin for decoding and encoding VP8 and VP9 streams | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vpx_plugin.svg)](https://hex.pm/api/packages/membrane_vpx_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vpx_plugin/) | | [membrane_abr_transcoder_plugin](https://github.com/membraneframework/membrane_abr_transcoder_plugin) | ABR (adaptive bitrate) transcoder, that accepts an h.264 video and outputs multiple variants of it with different qualities. | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_abr_transcoder_plugin.svg)](https://hex.pm/api/packages/membrane_abr_transcoder_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_abr_transcoder_plugin/) | +| [membrane_vk_video_plugin](https://github.com/membraneframework/membrane_vk_video_plugin) | Membrane H.264 decoder and encoder based on vk-video | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vk_video_plugin.svg)](https://hex.pm/api/packages/membrane_vk_video_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vk_video_plugin/) | | [membrane_h265_ffmpeg_plugin](https://github.com/gBillal/membrane_h265_ffmpeg_plugin) | [Maintainer: [gBillal](https://github.com/gBillal)] Membrane H265 decoder and encoder based on FFmpeg and x265 | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h265_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_h265_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h265_ffmpeg_plugin/) | | [elixir-turbojpeg](https://github.com/BinaryNoggin/elixir-turbojpeg) | [Maintainer: [BinaryNoggin](https://github.com/BinaryNoggin)] libjpeg-turbo bindings for Elixir | | | [membrane_subtitle_mixer_plugin](https://github.com/kim-company/membrane_subtitle_mixer_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that uses CEA708 to merge subtitles directly in H264 packets. | | #### Raw audio - | Package | Description | Links | | --- | --- | --- | | [membrane_raw_audio_parser_plugin](https://github.com/membraneframework/membrane_raw_audio_parser_plugin) | Membrane element for parsing raw audio | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_audio_parser_plugin.svg)](https://hex.pm/api/packages/membrane_raw_audio_parser_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_audio_parser_plugin/) | @@ -247,7 +248,6 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_audiometer_plugin](https://github.com/membraneframework/membrane_audiometer_plugin) | Elements for measuring the level of the audio stream | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_audiometer_plugin.svg)](https://hex.pm/api/packages/membrane_audiometer_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_audiometer_plugin/) | #### Raw video - | Package | Description | Links | | --- | --- | --- | | [membrane_raw_video_parser_plugin](https://github.com/membraneframework/membrane_raw_video_parser_plugin) | Membrane plugin for parsing raw video streams | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_video_parser_plugin.svg)](https://hex.pm/api/packages/membrane_raw_video_parser_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_video_parser_plugin/) | @@ -260,23 +260,19 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_overlay_plugin](https://github.com/membraneframework/membrane_overlay_plugin) | Filter for applying overlay image or text on top of video | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_overlay_plugin.svg)](https://hex.pm/api/packages/membrane_overlay_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_overlay_plugin/) | | [membrane_ffmpeg_swscale_plugin](https://github.com/membraneframework/membrane_ffmpeg_swscale_plugin) | Plugin providing an element scaling raw video frames, using SWScale module of FFmpeg library. | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_swscale_plugin.svg)](https://hex.pm/api/packages/membrane_ffmpeg_swscale_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_swscale_plugin/) | | [membrane_ffmpeg_video_filter_plugin](https://github.com/membraneframework/membrane_ffmpeg_video_filter_plugin) | FFmpeg-based video filters | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_video_filter_plugin.svg)](https://hex.pm/api/packages/membrane_ffmpeg_video_filter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_video_filter_plugin/) | -| [membrane_yolo_plugin](https://github.com/membraneframework/membrane_yolo_plugin) | Membrane Plugin for applying YOLO object detection on raw video frames | | -| [membrane_video_mixer_plugin](https://github.com/kim-company/membrane_video_mixer_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that mixes a variable number of input videos into one output using ffmpeg filters | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_video_mixer_plugin.svg)](https://hex.pm/api/packages/membrane_video_mixer_plugin) | +| [membrane_video_mixer_plugin](https://github.com/kim-company/membrane_video_mixer_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that mixes a variable number of input videos into one output using ffmpeg filters | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_video_mixer_plugin.svg)](https://hex.pm/api/packages/membrane_video_mixer_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_video_mixer_plugin/) | #### External APIs - | Package | Description | Links | | --- | --- | --- | | [membrane_aws_plugin](https://github.com/fishjam-dev/membrane_aws_plugin) | [Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] | | | [membrane_agora_plugin](https://github.com/membraneframework/membrane_agora_plugin) | Membrane Sink for Agora Server Gateway | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_agora_plugin.svg)](https://hex.pm/api/packages/membrane_agora_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_agora_plugin/) | -| [membrane_webrtc_live](https://github.com/membraneframework/membrane_webrtc_live) | | | | [membrane_element_gcloud_speech_to_text](https://github.com/membraneframework/membrane_element_gcloud_speech_to_text) | Membrane plugin providing speech recognition via Google Cloud Speech-to-Text API | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_element_gcloud_speech_to_text.svg)](https://hex.pm/api/packages/membrane_element_gcloud_speech_to_text) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_element_gcloud_speech_to_text/) | | [membrane_element_ibm_speech_to_text](https://github.com/membraneframework/membrane_element_ibm_speech_to_text) | Membrane plugin providing speech recognition via IBM Cloud Speech-to-Text service | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_element_ibm_speech_to_text.svg)](https://hex.pm/api/packages/membrane_element_ibm_speech_to_text) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_element_ibm_speech_to_text/) | | [membrane_s3_plugin](https://github.com/YuzuTen/membrane_s3_plugin) | [Maintainer: [YuzuTen](https://github.com/YuzuTen)] Membrane framework plugin to support S3 sources/destinations | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_s3_plugin.svg)](https://hex.pm/api/packages/membrane_s3_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_s3_plugin/) | | [membrane_transcription](https://github.com/lawik/membrane_transcription) | [Maintainer: [lawik](https://github.com/lawik)] Prototype transcription for Membrane | | ### Formats - | Package | Description | Links | | --- | --- | --- | | [membrane_rtp_format](https://github.com/membraneframework/membrane_rtp_format) | Real-time Transport Protocol format for Membrane Framework | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_format.svg)](https://hex.pm/api/packages/membrane_rtp_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_format/) | @@ -293,10 +289,10 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_vp8_format](https://github.com/membraneframework/membrane_vp8_format) | VP8 Membrane format | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vp8_format.svg)](https://hex.pm/api/packages/membrane_vp8_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vp8_format/) | | [membrane_vp9_format](https://github.com/membraneframework/membrane_vp9_format) | VP9 Membrane format | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vp9_format.svg)](https://hex.pm/api/packages/membrane_vp9_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vp9_format/) | | [membrane_g711_format](https://github.com/membraneframework/membrane_g711_format) | Membrane Multimedia Framework: G711 audio format definition | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_g711_format.svg)](https://hex.pm/api/packages/membrane_g711_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_g711_format/) | +| [membrane_av1_format](https://github.com/membraneframework/membrane_av1_format) | About Membrane Multimedia Framework: AV1 video format definition | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_av1_format.svg)](https://hex.pm/api/packages/membrane_av1_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_av1_format/) | | [membrane_h265_format](https://github.com/gBillal/membrane_h265_format) | [Maintainer: [gBillal](https://github.com/gBillal)] H265 video format definition | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h265_format.svg)](https://hex.pm/api/packages/membrane_h265_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h265_format/) | ### Standalone media libs - | Package | Description | Links | | --- | --- | --- | | [ex_webrtc](https://github.com/elixir-webrtc/ex_webrtc) | [Maintainer: [elixir-webrtc](https://github.com/elixir-webrtc)] An Elixir implementation of the W3C WebRTC API | [![Hex.pm](https://img.shields.io/hexpm/v/ex_webrtc.svg)](https://hex.pm/api/packages/ex_webrtc) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_webrtc/) | @@ -310,7 +306,6 @@ If you have any questions regarding Membrane Framework or need consulting, feel | [membrane_ffmpeg_generator](https://github.com/membraneframework-labs/membrane_ffmpeg_generator) | [Labs] FFmpeg video and audio generator for tests, benchmarks and demos. | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_generator.svg)](https://hex.pm/api/packages/membrane_ffmpeg_generator) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_generator/) | ### Utils - | Package | Description | Links | | --- | --- | --- | | [unifex](https://github.com/membraneframework/unifex) | Tool for generating interfaces between native C code and Elixir | [![Hex.pm](https://img.shields.io/hexpm/v/unifex.svg)](https://hex.pm/api/packages/unifex) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/unifex/) | diff --git a/guides/llms/packages_list.md b/guides/llms/packages_list.md new file mode 100644 index 000000000..99ffec24d --- /dev/null +++ b/guides/llms/packages_list.md @@ -0,0 +1,175 @@ + +### General +| Package | Description | +| --- | --- | +| [membrane_sdk](https://github.com/membraneframework/membrane_sdk) | Full power of Membrane in a single package | +| [membrane_core](https://github.com/membraneframework/membrane_core) | The core of Membrane Framework, multimedia processing framework written in Elixir | +| [membrane_rtc_engine](https://github.com/fishjam-dev/membrane_rtc_engine) | [Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] Customizable Real-time Communication Engine/SFU library focused on WebRTC. | +| [kino_membrane](https://github.com/membraneframework/kino_membrane) | Utilities for introspecting Membrane pipelines in Livebook | +| [docker_membrane](https://github.com/membraneframework-labs/docker_membrane) | [Labs] A docker image based on Ubuntu, with Erlang, Elixir and libraries necessary to test and run the Membrane Framework. | +| [membrane_demo](https://github.com/membraneframework/membrane_demo) | Examples of using the Membrane Framework | +| [membrane_tutorials](https://github.com/membraneframework/membrane_tutorials) | Repository which contains text and assets used in Membrane Framework tutorials. | +| [boombox](https://github.com/membraneframework/boombox) | Boombox is a simple streaming tool built on top of Membrane | + +### Plugins + +#### General purpose +| Package | Description | +| --- | --- | +| [membrane_file_plugin](https://github.com/membraneframework/membrane_file_plugin) | Membrane plugin for reading and writing to files | +| [membrane_hackney_plugin](https://github.com/membraneframework/membrane_hackney_plugin) | HTTP sink and source based on Hackney | +| [membrane_scissors_plugin](https://github.com/membraneframework/membrane_scissors_plugin) | Element for cutting off parts of the stream | +| [membrane_tee_plugin](https://github.com/membraneframework/membrane_tee_plugin) | Membrane plugin for splitting data from a single input to multiple outputs | +| [membrane_funnel_plugin](https://github.com/membraneframework/membrane_funnel_plugin) | Membrane plugin for merging multiple input streams into a single output | +| [membrane_realtimer_plugin](https://github.com/membraneframework/membrane_realtimer_plugin) | Membrane element limiting playback speed to realtime, according to buffers' timestamps | +| [membrane_stream_plugin](https://github.com/membraneframework/membrane_stream_plugin) | Plugin for recording the entire stream sent through Membrane pads into a binary format and replaying it | +| [membrane_fake_plugin](https://github.com/membraneframework/membrane_fake_plugin) | Fake Membrane sinks that drop incoming data | +| [membrane_pcap_plugin](https://github.com/membraneframework-labs/membrane_pcap_plugin) | [Labs] Membrane PCAP source, capable of reading captured packets in pcap format | +| [membrane_transcoder_plugin](https://github.com/membraneframework/membrane_transcoder_plugin) | Membrane plugin providing audio and video transcoding capabilities | +| [membrane_generator_plugin](https://github.com/membraneframework/membrane_generator_plugin) | Video and audio samples generator | +| [membrane_live_framerate_converter_plugin](https://github.com/kim-company/membrane_live_framerate_converter_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that drops or duplicates frames to match a target framerate. Designed for realtime applications | +| [membrane_template_plugin](https://github.com/membraneframework/membrane_template_plugin) | Template for Membrane Elements | + +#### AI +| Package | Description | +| --- | --- | +| [membrane_whisper_plugin](https://github.com/membraneframework/membrane_whisper_plugin) | Membrane plugin for integrating OpenAI's Whisper in audio processing pipelines | +| [membrane_yolo_plugin](https://github.com/membraneframework/membrane_yolo_plugin) | Membrane Plugin for applying YOLO object detection on raw video frames | + +#### Streaming protocols +| Package | Description | +| --- | --- | +| [membrane_webrtc_plugin](https://github.com/membraneframework/membrane_webrtc_plugin) | Plugin for streaming via WebRTC | +| [membrane_rtmp_plugin](https://github.com/membraneframework/membrane_rtmp_plugin) | RTMP server & client | +| [membrane_http_adaptive_stream_plugin](https://github.com/membraneframework/membrane_http_adaptive_stream_plugin) | Plugin generating manifests for HLS | +| [membrane_srt_plugin](https://github.com/membraneframework/membrane_srt_plugin) | | +| [membrane_udp_plugin](https://github.com/membraneframework/membrane_udp_plugin) | Membrane plugin for sending and receiving UDP streams | +| [membrane_tcp_plugin](https://github.com/membraneframework/membrane_tcp_plugin) | Membrane plugin for sending and receiving TCP streams | +| [membrane_rtp_plugin](https://github.com/membraneframework/membrane_rtp_plugin) | Membrane bins and elements for sending and receiving RTP/SRTP and RTCP/SRTCP streams | +| [membrane_rtp_h264_plugin](https://github.com/membraneframework/membrane_rtp_h264_plugin) | Membrane RTP payloader and depayloader for H264 | +| [membrane_rtp_aac_plugin](https://github.com/membraneframework/membrane_rtp_aac_plugin) | RTP AAC depayloader | +| [membrane_rtp_vp8_plugin](https://github.com/membraneframework/membrane_rtp_vp8_plugin) | Membrane elements for payloading and depayloading VP8 into RTP | +| [membrane_rtp_vp9_plugin](https://github.com/membraneframework-labs/membrane_rtp_vp9_plugin) | [Labs] Membrane elements for payloading and depayloading VP9 into RTP | +| [membrane_rtp_mpegaudio_plugin](https://github.com/membraneframework/membrane_rtp_mpegaudio_plugin) | Membrane RTP MPEG Audio depayloader | +| [membrane_rtp_opus_plugin](https://github.com/membraneframework/membrane_rtp_opus_plugin) | Membrane RTP payloader and depayloader for OPUS audio | +| [membrane_rtp_g711_plugin](https://github.com/membraneframework/membrane_rtp_g711_plugin) | Membrane RTP payloader and depayloader for G711 audio | +| [membrane_rtsp_plugin](https://github.com/gBillal/membrane_rtsp_plugin) | [Maintainer: [gBillal](https://github.com/gBillal)] Simplify connecting to RTSP server | +| [membrane_mpeg_ts_plugin](https://github.com/kim-company/membrane_mpeg_ts_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that demuxes MPEG-TS streams | +| [membrane_hls_plugin](https://github.com/kim-company/membrane_hls_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Adaptive live streaming plugin (HLS) for the Membrane Framework | + +#### Containers +| Package | Description | +| --- | --- | +| [membrane_mp4_plugin](https://github.com/membraneframework/membrane_mp4_plugin) | Utilities for MP4 container parsing and serialization and elements for muxing the stream to CMAF | +| [membrane_matroska_plugin](https://github.com/membraneframework/membrane_matroska_plugin) | Matroska muxer and demuxer | +| [membrane_flv_plugin](https://github.com/membraneframework/membrane_flv_plugin) | Muxer and demuxer elements for FLV format | +| [membrane_ivf_plugin](https://github.com/membraneframework/membrane_ivf_plugin) | Plugin for converting video stream into IVF format | +| [membrane_ogg_plugin](https://github.com/membraneframework/membrane_ogg_plugin) | Plugin for depayloading an Ogg file into an Opus stream | + +#### Audio codecs +| Package | Description | +| --- | --- | +| [membrane_aac_plugin](https://github.com/membraneframework/membrane_aac_plugin) | AAC parser and complementary elements for AAC codec | +| [membrane_aac_fdk_plugin](https://github.com/membraneframework/membrane_aac_fdk_plugin) | Membrane AAC decoder and encoder based on FDK library | +| [membrane_flac_plugin](https://github.com/membraneframework/membrane_flac_plugin) | Parser for files in FLAC bitstream format | +| [membrane_mp3_lame_plugin](https://github.com/membraneframework/membrane_mp3_lame_plugin) | Membrane MP3 encoder based on Lame | +| [membrane_mp3_mad_plugin](https://github.com/membraneframework/membrane_mp3_mad_plugin) | Membrane MP3 decoder based on MAD. | +| [membrane_opus_plugin](https://github.com/membraneframework/membrane_opus_plugin) | Membrane Opus encoder and decoder | +| [membrane_wav_plugin](https://github.com/membraneframework/membrane_wav_plugin) | Plugin providing elements handling audio in WAV file format. | +| [membrane_g711_plugin](https://github.com/membraneframework/membrane_g711_plugin) | Membrane G.711 decoder, encoder and parser | +| [membrane_g711_ffmpeg_plugin](https://github.com/membraneframework/membrane_g711_ffmpeg_plugin) | Membrane G.711 decoder and encoder based on FFmpeg | + +#### Video codecs +| Package | Description | +| --- | --- | +| [membrane_h26x_plugin](https://github.com/membraneframework/membrane_h26x_plugin) | Membrane h264 and h265 parsers | +| [membrane_h264_ffmpeg_plugin](https://github.com/membraneframework/membrane_h264_ffmpeg_plugin) | Membrane H264 decoder and encoder based on FFmpeg and x264 | +| [membrane_vpx_plugin](https://github.com/membraneframework/membrane_vpx_plugin) | Membrane plugin for decoding and encoding VP8 and VP9 streams | +| [membrane_abr_transcoder_plugin](https://github.com/membraneframework/membrane_abr_transcoder_plugin) | ABR (adaptive bitrate) transcoder, that accepts an h.264 video and outputs multiple variants of it with different qualities. | +| [membrane_vk_video_plugin](https://github.com/membraneframework/membrane_vk_video_plugin) | Membrane H.264 decoder and encoder based on vk-video | +| [membrane_h265_ffmpeg_plugin](https://github.com/gBillal/membrane_h265_ffmpeg_plugin) | [Maintainer: [gBillal](https://github.com/gBillal)] Membrane H265 decoder and encoder based on FFmpeg and x265 | +| [elixir-turbojpeg](https://github.com/BinaryNoggin/elixir-turbojpeg) | [Maintainer: [BinaryNoggin](https://github.com/BinaryNoggin)] libjpeg-turbo bindings for Elixir | +| [membrane_subtitle_mixer_plugin](https://github.com/kim-company/membrane_subtitle_mixer_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that uses CEA708 to merge subtitles directly in H264 packets. | + +#### Raw audio +| Package | Description | +| --- | --- | +| [membrane_raw_audio_parser_plugin](https://github.com/membraneframework/membrane_raw_audio_parser_plugin) | Membrane element for parsing raw audio | +| [membrane_portaudio_plugin](https://github.com/membraneframework/membrane_portaudio_plugin) | Raw audio retriever and player based on PortAudio | +| [membrane_audio_mix_plugin](https://github.com/membraneframework/membrane_audio_mix_plugin) | Plugin providing an element mixing raw audio frames. | +| [membrane_audio_filler_plugin](https://github.com/membraneframework/membrane_audio_filler_plugin) | Element for filling missing buffers in audio stream | +| [membrane_ffmpeg_swresample_plugin](https://github.com/membraneframework/membrane_ffmpeg_swresample_plugin) | Plugin performing audio conversion, resampling and channel mixing, using SWResample module of FFmpeg library | +| [membrane_audiometer_plugin](https://github.com/membraneframework/membrane_audiometer_plugin) | Elements for measuring the level of the audio stream | + +#### Raw video +| Package | Description | +| --- | --- | +| [membrane_raw_video_parser_plugin](https://github.com/membraneframework/membrane_raw_video_parser_plugin) | Membrane plugin for parsing raw video streams | +| [membrane_video_merger_plugin](https://github.com/membraneframework/membrane_video_merger_plugin) | Membrane raw video cutter, merger and cut & merge bin | +| [membrane_smelter_plugin](https://github.com/membraneframework/membrane_smelter_plugin) | Membrane plugin for video and audio mixing/compositing | +| [membrane_camera_capture_plugin](https://github.com/membraneframework/membrane_camera_capture_plugin) | A set of elements allowing for capturing local media such as camera or microphone | +| [membrane_rpicam_plugin](https://github.com/membraneframework/membrane_rpicam_plugin) | Membrane rpicam plugin | +| [membrane_framerate_converter_plugin](https://github.com/membraneframework/membrane_framerate_converter_plugin) | Element for converting frame rate of raw video stream | +| [membrane_sdl_plugin](https://github.com/membraneframework/membrane_sdl_plugin) | Membrane video player based on SDL | +| [membrane_overlay_plugin](https://github.com/membraneframework/membrane_overlay_plugin) | Filter for applying overlay image or text on top of video | +| [membrane_ffmpeg_swscale_plugin](https://github.com/membraneframework/membrane_ffmpeg_swscale_plugin) | Plugin providing an element scaling raw video frames, using SWScale module of FFmpeg library. | +| [membrane_ffmpeg_video_filter_plugin](https://github.com/membraneframework/membrane_ffmpeg_video_filter_plugin) | FFmpeg-based video filters | +| [membrane_video_mixer_plugin](https://github.com/kim-company/membrane_video_mixer_plugin) | [Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that mixes a variable number of input videos into one output using ffmpeg filters | + +#### External APIs +| Package | Description | +| --- | --- | +| [membrane_aws_plugin](https://github.com/fishjam-dev/membrane_aws_plugin) | [Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] | +| [membrane_agora_plugin](https://github.com/membraneframework/membrane_agora_plugin) | Membrane Sink for Agora Server Gateway | +| [membrane_element_gcloud_speech_to_text](https://github.com/membraneframework/membrane_element_gcloud_speech_to_text) | Membrane plugin providing speech recognition via Google Cloud Speech-to-Text API | +| [membrane_element_ibm_speech_to_text](https://github.com/membraneframework/membrane_element_ibm_speech_to_text) | Membrane plugin providing speech recognition via IBM Cloud Speech-to-Text service | +| [membrane_s3_plugin](https://github.com/YuzuTen/membrane_s3_plugin) | [Maintainer: [YuzuTen](https://github.com/YuzuTen)] Membrane framework plugin to support S3 sources/destinations | +| [membrane_transcription](https://github.com/lawik/membrane_transcription) | [Maintainer: [lawik](https://github.com/lawik)] Prototype transcription for Membrane | + +### Formats +| Package | Description | +| --- | --- | +| [membrane_rtp_format](https://github.com/membraneframework/membrane_rtp_format) | Real-time Transport Protocol format for Membrane Framework | +| [membrane_cmaf_format](https://github.com/membraneframework/membrane_cmaf_format) | Membrane description for Common Media Application Format | +| [membrane_matroska_format](https://github.com/membraneframework/membrane_matroska_format) | Matroska Membrane format | +| [membrane_mp4_format](https://github.com/membraneframework/membrane_mp4_format) | MPEG-4 container Membrane format | +| [membrane_raw_audio_format](https://github.com/membraneframework/membrane_raw_audio_format) | Raw audio format definition for the Membrane Multimedia Framework | +| [membrane_raw_video_format](https://github.com/membraneframework/membrane_raw_video_format) | Membrane Multimedia Framework: Raw video format definition | +| [membrane_aac_format](https://github.com/membraneframework/membrane_aac_format) | Advanced Audio Codec Membrane format | +| [membrane_opus_format](https://github.com/membraneframework/membrane_opus_format) | Opus audio format definition for Membrane Framework | +| [membrane_flac_format](https://github.com/membraneframework/membrane_flac_format) | FLAC audio format description for Membrane Framework | +| [membrane_mpegaudio_format](https://github.com/membraneframework/membrane_mpegaudio_format) | MPEG audio format definition for Membrane Framework | +| [membrane_h264_format](https://github.com/membraneframework/membrane_h264_format) | Membrane Multimedia Framework: H264 video format definition | +| [membrane_vp8_format](https://github.com/membraneframework/membrane_vp8_format) | VP8 Membrane format | +| [membrane_vp9_format](https://github.com/membraneframework/membrane_vp9_format) | VP9 Membrane format | +| [membrane_g711_format](https://github.com/membraneframework/membrane_g711_format) | Membrane Multimedia Framework: G711 audio format definition | +| [membrane_av1_format](https://github.com/membraneframework/membrane_av1_format) | About Membrane Multimedia Framework: AV1 video format definition | +| [membrane_h265_format](https://github.com/gBillal/membrane_h265_format) | [Maintainer: [gBillal](https://github.com/gBillal)] H265 video format definition | + +### Standalone media libs +| Package | Description | +| --- | --- | +| [ex_webrtc](https://github.com/elixir-webrtc/ex_webrtc) | [Maintainer: [elixir-webrtc](https://github.com/elixir-webrtc)] An Elixir implementation of the W3C WebRTC API | +| [ex_sdp](https://github.com/membraneframework/ex_sdp) | Parser and serializer for Session Description Protocol | +| [ex_libnice](https://github.com/membraneframework/ex_libnice) | Libnice-based Interactive Connectivity Establishment (ICE) protocol support for Elixir | +| [ex_libsrtp](https://github.com/membraneframework/ex_libsrtp) | Elixir bindings for libsrtp | +| [ex_m3u8](https://github.com/membraneframework/ex_m3u8) | Elixir package for serializing and deserializing M3U8 manifests. | +| [ex_hls](https://github.com/membraneframework/ex_hls) | An Elixir package for handling HLS streams | +| [ex_libsrt](https://github.com/membraneframework/ex_libsrt) | Elixir bindings to libsrt library exposing client and server APIs | +| [membrane_rtsp](https://github.com/membraneframework/membrane_rtsp) | RTSP client for Elixir | +| [membrane_ffmpeg_generator](https://github.com/membraneframework-labs/membrane_ffmpeg_generator) | [Labs] FFmpeg video and audio generator for tests, benchmarks and demos. | + +### Utils +| Package | Description | +| --- | --- | +| [unifex](https://github.com/membraneframework/unifex) | Tool for generating interfaces between native C code and Elixir | +| [bundlex](https://github.com/membraneframework/bundlex) | Multiplatform app bundler tool for Elixir | +| [beamchmark](https://github.com/membraneframework/beamchmark) | Elixir tool for benchmarking EVM performance | +| [bunch](https://github.com/membraneframework/bunch) | A bunch of helper functions, intended to make life easier | +| [bunch_native](https://github.com/membraneframework/bunch_native) | Native part of the Bunch package | +| [shmex](https://github.com/membraneframework/shmex) | Elixir bindings for shared memory | +| [membrane_timestamp_queue](https://github.com/membraneframework/membrane_timestamp_queue) | Queue that aligns streams from multiple sources basing on timestamps | +| [membrane_common_c](https://github.com/membraneframework/membrane_common_c) | Membrane Multimedia Framework: Common C Routines | +| [membrane_telemetry_metrics](https://github.com/membraneframework/membrane_telemetry_metrics) | Membrane tool for generating metrics | +| [membrane_opentelemetry](https://github.com/membraneframework-labs/membrane_opentelemetry) | [Labs] Utilities for using OpenTelemetry with Membrane | +| [membrane_precompiled_dependency_provider](https://github.com/membraneframework/membrane_precompiled_dependency_provider) | Provides URLs for precompiled dependencies used by Membrane plugins. | \ No newline at end of file diff --git a/guides/packages/00_General.md b/guides/packages/00_General.md index cd6cd9223..bf1fd037b 100644 --- a/guides/packages/00_General.md +++ b/guides/packages/00_General.md @@ -1,41 +1,41 @@ ## membrane_sdk -Full power of Membrane in a single package +Full power of Membrane in a single package [![Hex.pm](https://img.shields.io/hexpm/v/membrane_sdk.svg)](https://hex.pm/api/packages/membrane_sdk) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_sdk/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_sdk) - + ## membrane_core -The core of Membrane Framework, multimedia processing framework written in Elixir +The core of Membrane Framework, multimedia processing framework written in Elixir [![Hex.pm](https://img.shields.io/hexpm/v/membrane_core.svg)](https://hex.pm/api/packages/membrane_core) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_core/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_core) - + ## membrane_rtc_engine -[Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] Customizable Real-time Communication Engine/SFU library focused on WebRTC. +[Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] Customizable Real-time Communication Engine/SFU library focused on WebRTC. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtc_engine.svg)](https://hex.pm/api/packages/membrane_rtc_engine) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtc_engine/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/fishjam-dev/membrane_rtc_engine) - + ## kino_membrane -Utilities for introspecting Membrane pipelines in Livebook +Utilities for introspecting Membrane pipelines in Livebook [![Hex.pm](https://img.shields.io/hexpm/v/kino_membrane.svg)](https://hex.pm/api/packages/kino_membrane) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/kino_membrane/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/kino_membrane) - + ## docker_membrane -[Labs] A docker image based on Ubuntu, with Erlang, Elixir and libraries necessary to test and run the Membrane Framework. +[Labs] A docker image based on Ubuntu, with Erlang, Elixir and libraries necessary to test and run the Membrane Framework. [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework-labs/docker_membrane) - + ## membrane_demo -Examples of using the Membrane Framework +Examples of using the Membrane Framework [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_demo) - + ## membrane_tutorials -Repository which contains text and assets used in Membrane Framework tutorials. +Repository which contains text and assets used in Membrane Framework tutorials. [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_tutorials) - + ## boombox -Boombox is a simple streaming tool built on top of Membrane +Boombox is a simple streaming tool built on top of Membrane [![Hex.pm](https://img.shields.io/hexpm/v/boombox.svg)](https://hex.pm/api/packages/boombox) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/boombox/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/boombox) - + diff --git a/guides/packages/02_Plugins_|_General_purpose.md b/guides/packages/02_Plugins_|_General_purpose.md index 59f96bf86..c51884878 100644 --- a/guides/packages/02_Plugins_|_General_purpose.md +++ b/guides/packages/02_Plugins_|_General_purpose.md @@ -1,66 +1,66 @@ ## membrane_file_plugin -Membrane plugin for reading and writing to files +Membrane plugin for reading and writing to files [![Hex.pm](https://img.shields.io/hexpm/v/membrane_file_plugin.svg)](https://hex.pm/api/packages/membrane_file_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_file_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_file_plugin) - + ## membrane_hackney_plugin -HTTP sink and source based on Hackney +HTTP sink and source based on Hackney [![Hex.pm](https://img.shields.io/hexpm/v/membrane_hackney_plugin.svg)](https://hex.pm/api/packages/membrane_hackney_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_hackney_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_hackney_plugin) - + ## membrane_scissors_plugin -Element for cutting off parts of the stream +Element for cutting off parts of the stream [![Hex.pm](https://img.shields.io/hexpm/v/membrane_scissors_plugin.svg)](https://hex.pm/api/packages/membrane_scissors_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_scissors_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_scissors_plugin) - + ## membrane_tee_plugin -Membrane plugin for splitting data from a single input to multiple outputs +Membrane plugin for splitting data from a single input to multiple outputs [![Hex.pm](https://img.shields.io/hexpm/v/membrane_tee_plugin.svg)](https://hex.pm/api/packages/membrane_tee_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_tee_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_tee_plugin) - + ## membrane_funnel_plugin -Membrane plugin for merging multiple input streams into a single output +Membrane plugin for merging multiple input streams into a single output [![Hex.pm](https://img.shields.io/hexpm/v/membrane_funnel_plugin.svg)](https://hex.pm/api/packages/membrane_funnel_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_funnel_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_funnel_plugin) - + ## membrane_realtimer_plugin -Membrane element limiting playback speed to realtime, according to buffers' timestamps +Membrane element limiting playback speed to realtime, according to buffers' timestamps [![Hex.pm](https://img.shields.io/hexpm/v/membrane_realtimer_plugin.svg)](https://hex.pm/api/packages/membrane_realtimer_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_realtimer_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_realtimer_plugin) - + ## membrane_stream_plugin -Plugin for recording the entire stream sent through Membrane pads into a binary format and replaying it +Plugin for recording the entire stream sent through Membrane pads into a binary format and replaying it [![Hex.pm](https://img.shields.io/hexpm/v/membrane_stream_plugin.svg)](https://hex.pm/api/packages/membrane_stream_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_stream_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_stream_plugin) - + ## membrane_fake_plugin -Fake Membrane sinks that drop incoming data +Fake Membrane sinks that drop incoming data [![Hex.pm](https://img.shields.io/hexpm/v/membrane_fake_plugin.svg)](https://hex.pm/api/packages/membrane_fake_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_fake_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_fake_plugin) - + ## membrane_pcap_plugin -[Labs] Membrane PCAP source, capable of reading captured packets in pcap format +[Labs] Membrane PCAP source, capable of reading captured packets in pcap format [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework-labs/membrane_pcap_plugin) - + ## membrane_transcoder_plugin -Membrane plugin providing audio and video transcoding capabilities +Membrane plugin providing audio and video transcoding capabilities [![Hex.pm](https://img.shields.io/hexpm/v/membrane_transcoder_plugin.svg)](https://hex.pm/api/packages/membrane_transcoder_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_transcoder_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_transcoder_plugin) - + ## membrane_generator_plugin -Video and audio samples generator +Video and audio samples generator [![Hex.pm](https://img.shields.io/hexpm/v/membrane_generator_plugin.svg)](https://hex.pm/api/packages/membrane_generator_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_generator_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_generator_plugin) - + ## membrane_live_framerate_converter_plugin -[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that drops or duplicates frames to match a target framerate. Designed for realtime applications +[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that drops or duplicates frames to match a target framerate. Designed for realtime applications + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_live_framerate_converter_plugin.svg)](https://hex.pm/api/packages/membrane_live_framerate_converter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_live_framerate_converter_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_live_framerate_converter_plugin) - [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_live_framerate_converter_plugin) - ## membrane_template_plugin -Template for Membrane Elements +Template for Membrane Elements [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_template_plugin) - + diff --git a/guides/packages/03_Plugins_|_AI.md b/guides/packages/03_Plugins_|_AI.md new file mode 100644 index 000000000..e3a9cdea6 --- /dev/null +++ b/guides/packages/03_Plugins_|_AI.md @@ -0,0 +1,11 @@ + +## membrane_whisper_plugin +Membrane plugin for integrating OpenAI's Whisper in audio processing pipelines + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_whisper_plugin.svg)](https://hex.pm/api/packages/membrane_whisper_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_whisper_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_whisper_plugin) + +## membrane_yolo_plugin +Membrane Plugin for applying YOLO object detection on raw video frames + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_yolo_plugin.svg)](https://hex.pm/api/packages/membrane_yolo_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_yolo_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_yolo_plugin) + diff --git a/guides/packages/03_Plugins_|_Streaming_protocols.md b/guides/packages/04_Plugins_|_Streaming_protocols.md similarity index 91% rename from guides/packages/03_Plugins_|_Streaming_protocols.md rename to guides/packages/04_Plugins_|_Streaming_protocols.md index b2ba583d7..8c982275d 100644 --- a/guides/packages/03_Plugins_|_Streaming_protocols.md +++ b/guides/packages/04_Plugins_|_Streaming_protocols.md @@ -1,86 +1,86 @@ ## membrane_webrtc_plugin -Plugin for streaming via WebRTC +Plugin for streaming via WebRTC [![Hex.pm](https://img.shields.io/hexpm/v/membrane_webrtc_plugin.svg)](https://hex.pm/api/packages/membrane_webrtc_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_webrtc_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_webrtc_plugin) - + ## membrane_rtmp_plugin -RTMP server & client +RTMP server & client [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtmp_plugin.svg)](https://hex.pm/api/packages/membrane_rtmp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtmp_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtmp_plugin) - + ## membrane_http_adaptive_stream_plugin -Plugin generating manifests for HLS +Plugin generating manifests for HLS [![Hex.pm](https://img.shields.io/hexpm/v/membrane_http_adaptive_stream_plugin.svg)](https://hex.pm/api/packages/membrane_http_adaptive_stream_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_http_adaptive_stream_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_http_adaptive_stream_plugin) - + ## membrane_srt_plugin - + [![Hex.pm](https://img.shields.io/hexpm/v/membrane_srt_plugin.svg)](https://hex.pm/api/packages/membrane_srt_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_srt_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_srt_plugin) - + ## membrane_udp_plugin -Membrane plugin for sending and receiving UDP streams +Membrane plugin for sending and receiving UDP streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_udp_plugin.svg)](https://hex.pm/api/packages/membrane_udp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_udp_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_udp_plugin) - + ## membrane_tcp_plugin -Membrane plugin for sending and receiving TCP streams +Membrane plugin for sending and receiving TCP streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_tcp_plugin.svg)](https://hex.pm/api/packages/membrane_tcp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_tcp_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_tcp_plugin) - + ## membrane_rtp_plugin -Membrane bins and elements for sending and receiving RTP/SRTP and RTCP/SRTCP streams +Membrane bins and elements for sending and receiving RTP/SRTP and RTCP/SRTCP streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_plugin) - + ## membrane_rtp_h264_plugin -Membrane RTP payloader and depayloader for H264 +Membrane RTP payloader and depayloader for H264 [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_h264_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_h264_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_h264_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_h264_plugin) - + ## membrane_rtp_aac_plugin -RTP AAC depayloader +RTP AAC depayloader [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_aac_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_aac_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_aac_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_aac_plugin) - + ## membrane_rtp_vp8_plugin -Membrane elements for payloading and depayloading VP8 into RTP +Membrane elements for payloading and depayloading VP8 into RTP [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_vp8_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_vp8_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_vp8_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_vp8_plugin) - + ## membrane_rtp_vp9_plugin -[Labs] Membrane elements for payloading and depayloading VP9 into RTP +[Labs] Membrane elements for payloading and depayloading VP9 into RTP [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework-labs/membrane_rtp_vp9_plugin) - + ## membrane_rtp_mpegaudio_plugin -Membrane RTP MPEG Audio depayloader +Membrane RTP MPEG Audio depayloader [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_mpegaudio_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_mpegaudio_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_mpegaudio_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_mpegaudio_plugin) - + ## membrane_rtp_opus_plugin -Membrane RTP payloader and depayloader for OPUS audio +Membrane RTP payloader and depayloader for OPUS audio [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_opus_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_opus_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_opus_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_opus_plugin) - + ## membrane_rtp_g711_plugin -Membrane RTP payloader and depayloader for G711 audio +Membrane RTP payloader and depayloader for G711 audio [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_g711_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_g711_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_g711_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_g711_plugin) - + ## membrane_rtsp_plugin -[Maintainer: [gBillal](https://github.com/gBillal)] Simplify connecting to RTSP server +[Maintainer: [gBillal](https://github.com/gBillal)] Simplify connecting to RTSP server [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtsp_plugin.svg)](https://hex.pm/api/packages/membrane_rtsp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtsp_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/gBillal/membrane_rtsp_plugin) - + ## membrane_mpeg_ts_plugin -[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that demuxes MPEG-TS streams +[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that demuxes MPEG-TS streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mpeg_ts_plugin.svg)](https://hex.pm/api/packages/membrane_mpeg_ts_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mpeg_ts_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_mpeg_ts_plugin) - + ## membrane_hls_plugin -[Maintainer: [kim-company](https://github.com/kim-company)] Adaptive live streaming plugin (HLS) for the Membrane Framework +[Maintainer: [kim-company](https://github.com/kim-company)] Adaptive live streaming plugin (HLS) for the Membrane Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_hls_plugin.svg)](https://hex.pm/api/packages/membrane_hls_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_hls_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_hls_plugin) - + diff --git a/guides/packages/04_Plugins_|_Containers.md b/guides/packages/05_Plugins_|_Containers.md similarity index 90% rename from guides/packages/04_Plugins_|_Containers.md rename to guides/packages/05_Plugins_|_Containers.md index e049cd52d..b89129b9a 100644 --- a/guides/packages/04_Plugins_|_Containers.md +++ b/guides/packages/05_Plugins_|_Containers.md @@ -1,26 +1,26 @@ ## membrane_mp4_plugin -Utilities for MP4 container parsing and serialization and elements for muxing the stream to CMAF +Utilities for MP4 container parsing and serialization and elements for muxing the stream to CMAF [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mp4_plugin.svg)](https://hex.pm/api/packages/membrane_mp4_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mp4_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_mp4_plugin) - + ## membrane_matroska_plugin -Matroska muxer and demuxer +Matroska muxer and demuxer [![Hex.pm](https://img.shields.io/hexpm/v/membrane_matroska_plugin.svg)](https://hex.pm/api/packages/membrane_matroska_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_matroska_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_matroska_plugin) - + ## membrane_flv_plugin -Muxer and demuxer elements for FLV format +Muxer and demuxer elements for FLV format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_flv_plugin.svg)](https://hex.pm/api/packages/membrane_flv_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_flv_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_flv_plugin) - + ## membrane_ivf_plugin -Plugin for converting video stream into IVF format +Plugin for converting video stream into IVF format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ivf_plugin.svg)](https://hex.pm/api/packages/membrane_ivf_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ivf_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_ivf_plugin) - + ## membrane_ogg_plugin -Plugin for depayloading an Ogg file into an Opus stream +Plugin for depayloading an Ogg file into an Opus stream [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ogg_plugin.svg)](https://hex.pm/api/packages/membrane_ogg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ogg_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_ogg_plugin) - + diff --git a/guides/packages/05_Plugins_|_Audio_codecs.md b/guides/packages/06_Plugins_|_Audio_codecs.md similarity index 89% rename from guides/packages/05_Plugins_|_Audio_codecs.md rename to guides/packages/06_Plugins_|_Audio_codecs.md index b41d4ead9..5a26dbb7a 100644 --- a/guides/packages/05_Plugins_|_Audio_codecs.md +++ b/guides/packages/06_Plugins_|_Audio_codecs.md @@ -1,46 +1,46 @@ ## membrane_aac_plugin -AAC parser and complementary elements for AAC codec +AAC parser and complementary elements for AAC codec [![Hex.pm](https://img.shields.io/hexpm/v/membrane_aac_plugin.svg)](https://hex.pm/api/packages/membrane_aac_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_aac_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_aac_plugin) - + ## membrane_aac_fdk_plugin -Membrane AAC decoder and encoder based on FDK library +Membrane AAC decoder and encoder based on FDK library [![Hex.pm](https://img.shields.io/hexpm/v/membrane_aac_fdk_plugin.svg)](https://hex.pm/api/packages/membrane_aac_fdk_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_aac_fdk_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_aac_fdk_plugin) - + ## membrane_flac_plugin -Parser for files in FLAC bitstream format +Parser for files in FLAC bitstream format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_flac_plugin.svg)](https://hex.pm/api/packages/membrane_flac_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_flac_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_flac_plugin) - + ## membrane_mp3_lame_plugin -Membrane MP3 encoder based on Lame +Membrane MP3 encoder based on Lame [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mp3_lame_plugin.svg)](https://hex.pm/api/packages/membrane_mp3_lame_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mp3_lame_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_mp3_lame_plugin) - + ## membrane_mp3_mad_plugin -Membrane MP3 decoder based on MAD. +Membrane MP3 decoder based on MAD. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mp3_mad_plugin.svg)](https://hex.pm/api/packages/membrane_mp3_mad_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mp3_mad_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_mp3_mad_plugin) - + ## membrane_opus_plugin -Membrane Opus encoder and decoder +Membrane Opus encoder and decoder [![Hex.pm](https://img.shields.io/hexpm/v/membrane_opus_plugin.svg)](https://hex.pm/api/packages/membrane_opus_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_opus_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_opus_plugin) - + ## membrane_wav_plugin -Plugin providing elements handling audio in WAV file format. +Plugin providing elements handling audio in WAV file format. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_wav_plugin.svg)](https://hex.pm/api/packages/membrane_wav_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_wav_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_wav_plugin) - + ## membrane_g711_plugin -Membrane G.711 decoder, encoder and parser +Membrane G.711 decoder, encoder and parser [![Hex.pm](https://img.shields.io/hexpm/v/membrane_g711_plugin.svg)](https://hex.pm/api/packages/membrane_g711_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_g711_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_g711_plugin) - + ## membrane_g711_ffmpeg_plugin -Membrane G.711 decoder and encoder based on FFmpeg +Membrane G.711 decoder and encoder based on FFmpeg [![Hex.pm](https://img.shields.io/hexpm/v/membrane_g711_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_g711_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_g711_ffmpeg_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_g711_ffmpeg_plugin) - + diff --git a/guides/packages/06_Plugins_|_Video_codecs.md b/guides/packages/07_Plugins_|_Video_codecs.md similarity index 78% rename from guides/packages/06_Plugins_|_Video_codecs.md rename to guides/packages/07_Plugins_|_Video_codecs.md index dae69e196..297c08790 100644 --- a/guides/packages/06_Plugins_|_Video_codecs.md +++ b/guides/packages/07_Plugins_|_Video_codecs.md @@ -1,36 +1,41 @@ ## membrane_h26x_plugin -Membrane h264 and h265 parsers +Membrane h264 and h265 parsers [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h26x_plugin.svg)](https://hex.pm/api/packages/membrane_h26x_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h26x_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_h26x_plugin) - + ## membrane_h264_ffmpeg_plugin -Membrane H264 decoder and encoder based on FFmpeg and x264 +Membrane H264 decoder and encoder based on FFmpeg and x264 [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h264_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_h264_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h264_ffmpeg_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_h264_ffmpeg_plugin) - + ## membrane_vpx_plugin -Membrane plugin for decoding and encoding VP8 and VP9 streams +Membrane plugin for decoding and encoding VP8 and VP9 streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vpx_plugin.svg)](https://hex.pm/api/packages/membrane_vpx_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vpx_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_vpx_plugin) - + ## membrane_abr_transcoder_plugin -ABR (adaptive bitrate) transcoder, that accepts an h.264 video and outputs multiple variants of it with different qualities. +ABR (adaptive bitrate) transcoder, that accepts an h.264 video and outputs multiple variants of it with different qualities. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_abr_transcoder_plugin.svg)](https://hex.pm/api/packages/membrane_abr_transcoder_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_abr_transcoder_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_abr_transcoder_plugin) - + +## membrane_vk_video_plugin +Membrane H.264 decoder and encoder based on vk-video + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_vk_video_plugin.svg)](https://hex.pm/api/packages/membrane_vk_video_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vk_video_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_vk_video_plugin) + ## membrane_h265_ffmpeg_plugin -[Maintainer: [gBillal](https://github.com/gBillal)] Membrane H265 decoder and encoder based on FFmpeg and x265 +[Maintainer: [gBillal](https://github.com/gBillal)] Membrane H265 decoder and encoder based on FFmpeg and x265 [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h265_ffmpeg_plugin.svg)](https://hex.pm/api/packages/membrane_h265_ffmpeg_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h265_ffmpeg_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/gBillal/membrane_h265_ffmpeg_plugin) - + ## elixir-turbojpeg -[Maintainer: [BinaryNoggin](https://github.com/BinaryNoggin)] libjpeg-turbo bindings for Elixir +[Maintainer: [BinaryNoggin](https://github.com/BinaryNoggin)] libjpeg-turbo bindings for Elixir [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/BinaryNoggin/elixir-turbojpeg) - + ## membrane_subtitle_mixer_plugin -[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that uses CEA708 to merge subtitles directly in H264 packets. +[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that uses CEA708 to merge subtitles directly in H264 packets. [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_subtitle_mixer_plugin) - + diff --git a/guides/packages/07_Plugins_|_Raw_audio.md b/guides/packages/08_Plugins_|_Raw_audio.md similarity index 89% rename from guides/packages/07_Plugins_|_Raw_audio.md rename to guides/packages/08_Plugins_|_Raw_audio.md index ca529d682..a9d4b3e88 100644 --- a/guides/packages/07_Plugins_|_Raw_audio.md +++ b/guides/packages/08_Plugins_|_Raw_audio.md @@ -1,31 +1,31 @@ ## membrane_raw_audio_parser_plugin -Membrane element for parsing raw audio +Membrane element for parsing raw audio [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_audio_parser_plugin.svg)](https://hex.pm/api/packages/membrane_raw_audio_parser_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_audio_parser_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_raw_audio_parser_plugin) - + ## membrane_portaudio_plugin -Raw audio retriever and player based on PortAudio +Raw audio retriever and player based on PortAudio [![Hex.pm](https://img.shields.io/hexpm/v/membrane_portaudio_plugin.svg)](https://hex.pm/api/packages/membrane_portaudio_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_portaudio_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_portaudio_plugin) - + ## membrane_audio_mix_plugin -Plugin providing an element mixing raw audio frames. +Plugin providing an element mixing raw audio frames. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_audio_mix_plugin.svg)](https://hex.pm/api/packages/membrane_audio_mix_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_audio_mix_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_audio_mix_plugin) - + ## membrane_audio_filler_plugin -Element for filling missing buffers in audio stream +Element for filling missing buffers in audio stream [![Hex.pm](https://img.shields.io/hexpm/v/membrane_audio_filler_plugin.svg)](https://hex.pm/api/packages/membrane_audio_filler_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_audio_filler_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_audio_filler_plugin) - + ## membrane_ffmpeg_swresample_plugin -Plugin performing audio conversion, resampling and channel mixing, using SWResample module of FFmpeg library +Plugin performing audio conversion, resampling and channel mixing, using SWResample module of FFmpeg library [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_swresample_plugin.svg)](https://hex.pm/api/packages/membrane_ffmpeg_swresample_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_swresample_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_ffmpeg_swresample_plugin) - + ## membrane_audiometer_plugin -Elements for measuring the level of the audio stream +Elements for measuring the level of the audio stream [![Hex.pm](https://img.shields.io/hexpm/v/membrane_audiometer_plugin.svg)](https://hex.pm/api/packages/membrane_audiometer_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_audiometer_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_audiometer_plugin) - + diff --git a/guides/packages/08_Plugins_|_Raw_video.md b/guides/packages/09_Plugins_|_Raw_video.md similarity index 85% rename from guides/packages/08_Plugins_|_Raw_video.md rename to guides/packages/09_Plugins_|_Raw_video.md index e821be236..6deeeec2f 100644 --- a/guides/packages/08_Plugins_|_Raw_video.md +++ b/guides/packages/09_Plugins_|_Raw_video.md @@ -1,61 +1,56 @@ ## membrane_raw_video_parser_plugin -Membrane plugin for parsing raw video streams +Membrane plugin for parsing raw video streams [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_video_parser_plugin.svg)](https://hex.pm/api/packages/membrane_raw_video_parser_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_video_parser_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_raw_video_parser_plugin) - + ## membrane_video_merger_plugin -Membrane raw video cutter, merger and cut & merge bin +Membrane raw video cutter, merger and cut & merge bin [![Hex.pm](https://img.shields.io/hexpm/v/membrane_video_merger_plugin.svg)](https://hex.pm/api/packages/membrane_video_merger_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_video_merger_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_video_merger_plugin) - + ## membrane_smelter_plugin -Membrane plugin for video and audio mixing/compositing +Membrane plugin for video and audio mixing/compositing [![Hex.pm](https://img.shields.io/hexpm/v/membrane_smelter_plugin.svg)](https://hex.pm/api/packages/membrane_smelter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_smelter_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_smelter_plugin) - + ## membrane_camera_capture_plugin -A set of elements allowing for capturing local media such as camera or microphone +A set of elements allowing for capturing local media such as camera or microphone [![Hex.pm](https://img.shields.io/hexpm/v/membrane_camera_capture_plugin.svg)](https://hex.pm/api/packages/membrane_camera_capture_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_camera_capture_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_camera_capture_plugin) - + ## membrane_rpicam_plugin -Membrane rpicam plugin +Membrane rpicam plugin [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rpicam_plugin.svg)](https://hex.pm/api/packages/membrane_rpicam_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rpicam_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rpicam_plugin) - + ## membrane_framerate_converter_plugin -Element for converting frame rate of raw video stream +Element for converting frame rate of raw video stream [![Hex.pm](https://img.shields.io/hexpm/v/membrane_framerate_converter_plugin.svg)](https://hex.pm/api/packages/membrane_framerate_converter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_framerate_converter_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_framerate_converter_plugin) - + ## membrane_sdl_plugin -Membrane video player based on SDL +Membrane video player based on SDL [![Hex.pm](https://img.shields.io/hexpm/v/membrane_sdl_plugin.svg)](https://hex.pm/api/packages/membrane_sdl_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_sdl_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_sdl_plugin) - + ## membrane_overlay_plugin -Filter for applying overlay image or text on top of video +Filter for applying overlay image or text on top of video [![Hex.pm](https://img.shields.io/hexpm/v/membrane_overlay_plugin.svg)](https://hex.pm/api/packages/membrane_overlay_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_overlay_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_overlay_plugin) - + ## membrane_ffmpeg_swscale_plugin -Plugin providing an element scaling raw video frames, using SWScale module of FFmpeg library. +Plugin providing an element scaling raw video frames, using SWScale module of FFmpeg library. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_swscale_plugin.svg)](https://hex.pm/api/packages/membrane_ffmpeg_swscale_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_swscale_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_ffmpeg_swscale_plugin) - + ## membrane_ffmpeg_video_filter_plugin -FFmpeg-based video filters +FFmpeg-based video filters [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_video_filter_plugin.svg)](https://hex.pm/api/packages/membrane_ffmpeg_video_filter_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_video_filter_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_ffmpeg_video_filter_plugin) - -## membrane_yolo_plugin -Membrane Plugin for applying YOLO object detection on raw video frames - [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_yolo_plugin) - ## membrane_video_mixer_plugin -[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that mixes a variable number of input videos into one output using ffmpeg filters +[Maintainer: [kim-company](https://github.com/kim-company)] Membrane.Filter that mixes a variable number of input videos into one output using ffmpeg filters + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_video_mixer_plugin.svg)](https://hex.pm/api/packages/membrane_video_mixer_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_video_mixer_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_video_mixer_plugin) -[![Hex.pm](https://img.shields.io/hexpm/v/membrane_video_mixer_plugin.svg)](https://hex.pm/api/packages/membrane_video_mixer_plugin) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/kim-company/membrane_video_mixer_plugin) - diff --git a/guides/packages/09_Plugins_|_External_APIs.md b/guides/packages/10_Plugins_|_External_APIs.md similarity index 85% rename from guides/packages/09_Plugins_|_External_APIs.md rename to guides/packages/10_Plugins_|_External_APIs.md index f37225ed9..068017bca 100644 --- a/guides/packages/09_Plugins_|_External_APIs.md +++ b/guides/packages/10_Plugins_|_External_APIs.md @@ -1,36 +1,31 @@ ## membrane_aws_plugin -[Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] +[Maintainer: [fishjam-dev](https://github.com/fishjam-dev)] [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/fishjam-dev/membrane_aws_plugin) - + ## membrane_agora_plugin -Membrane Sink for Agora Server Gateway +Membrane Sink for Agora Server Gateway [![Hex.pm](https://img.shields.io/hexpm/v/membrane_agora_plugin.svg)](https://hex.pm/api/packages/membrane_agora_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_agora_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_agora_plugin) - -## membrane_webrtc_live - - [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_webrtc_live) - ## membrane_element_gcloud_speech_to_text -Membrane plugin providing speech recognition via Google Cloud Speech-to-Text API +Membrane plugin providing speech recognition via Google Cloud Speech-to-Text API [![Hex.pm](https://img.shields.io/hexpm/v/membrane_element_gcloud_speech_to_text.svg)](https://hex.pm/api/packages/membrane_element_gcloud_speech_to_text) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_element_gcloud_speech_to_text/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_element_gcloud_speech_to_text) - + ## membrane_element_ibm_speech_to_text -Membrane plugin providing speech recognition via IBM Cloud Speech-to-Text service +Membrane plugin providing speech recognition via IBM Cloud Speech-to-Text service [![Hex.pm](https://img.shields.io/hexpm/v/membrane_element_ibm_speech_to_text.svg)](https://hex.pm/api/packages/membrane_element_ibm_speech_to_text) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_element_ibm_speech_to_text/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_element_ibm_speech_to_text) - + ## membrane_s3_plugin -[Maintainer: [YuzuTen](https://github.com/YuzuTen)] Membrane framework plugin to support S3 sources/destinations +[Maintainer: [YuzuTen](https://github.com/YuzuTen)] Membrane framework plugin to support S3 sources/destinations [![Hex.pm](https://img.shields.io/hexpm/v/membrane_s3_plugin.svg)](https://hex.pm/api/packages/membrane_s3_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_s3_plugin/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/YuzuTen/membrane_s3_plugin) - + ## membrane_transcription -[Maintainer: [lawik](https://github.com/lawik)] Prototype transcription for Membrane +[Maintainer: [lawik](https://github.com/lawik)] Prototype transcription for Membrane [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/lawik/membrane_transcription) - + diff --git a/guides/packages/10_Formats.md b/guides/packages/11_Formats.md similarity index 85% rename from guides/packages/10_Formats.md rename to guides/packages/11_Formats.md index 050c13954..a6286309b 100644 --- a/guides/packages/10_Formats.md +++ b/guides/packages/11_Formats.md @@ -1,76 +1,81 @@ ## membrane_rtp_format -Real-time Transport Protocol format for Membrane Framework +Real-time Transport Protocol format for Membrane Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_format.svg)](https://hex.pm/api/packages/membrane_rtp_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtp_format) - + ## membrane_cmaf_format -Membrane description for Common Media Application Format +Membrane description for Common Media Application Format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_cmaf_format.svg)](https://hex.pm/api/packages/membrane_cmaf_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_cmaf_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_cmaf_format) - + ## membrane_matroska_format -Matroska Membrane format +Matroska Membrane format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_matroska_format.svg)](https://hex.pm/api/packages/membrane_matroska_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_matroska_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_matroska_format) - + ## membrane_mp4_format -MPEG-4 container Membrane format +MPEG-4 container Membrane format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mp4_format.svg)](https://hex.pm/api/packages/membrane_mp4_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mp4_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_mp4_format) - + ## membrane_raw_audio_format -Raw audio format definition for the Membrane Multimedia Framework +Raw audio format definition for the Membrane Multimedia Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_audio_format.svg)](https://hex.pm/api/packages/membrane_raw_audio_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_audio_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_raw_audio_format) - + ## membrane_raw_video_format -Membrane Multimedia Framework: Raw video format definition +Membrane Multimedia Framework: Raw video format definition [![Hex.pm](https://img.shields.io/hexpm/v/membrane_raw_video_format.svg)](https://hex.pm/api/packages/membrane_raw_video_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_raw_video_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_raw_video_format) - + ## membrane_aac_format -Advanced Audio Codec Membrane format +Advanced Audio Codec Membrane format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_aac_format.svg)](https://hex.pm/api/packages/membrane_aac_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_aac_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_aac_format) - + ## membrane_opus_format -Opus audio format definition for Membrane Framework +Opus audio format definition for Membrane Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_opus_format.svg)](https://hex.pm/api/packages/membrane_opus_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_opus_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_opus_format) - + ## membrane_flac_format -FLAC audio format description for Membrane Framework +FLAC audio format description for Membrane Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_flac_format.svg)](https://hex.pm/api/packages/membrane_flac_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_flac_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_flac_format) - + ## membrane_mpegaudio_format -MPEG audio format definition for Membrane Framework +MPEG audio format definition for Membrane Framework [![Hex.pm](https://img.shields.io/hexpm/v/membrane_mpegaudio_format.svg)](https://hex.pm/api/packages/membrane_mpegaudio_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_mpegaudio_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_mpegaudio_format) - + ## membrane_h264_format -Membrane Multimedia Framework: H264 video format definition +Membrane Multimedia Framework: H264 video format definition [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h264_format.svg)](https://hex.pm/api/packages/membrane_h264_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h264_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_h264_format) - + ## membrane_vp8_format -VP8 Membrane format +VP8 Membrane format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vp8_format.svg)](https://hex.pm/api/packages/membrane_vp8_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vp8_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_vp8_format) - + ## membrane_vp9_format -VP9 Membrane format +VP9 Membrane format [![Hex.pm](https://img.shields.io/hexpm/v/membrane_vp9_format.svg)](https://hex.pm/api/packages/membrane_vp9_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_vp9_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_vp9_format) - + ## membrane_g711_format -Membrane Multimedia Framework: G711 audio format definition +Membrane Multimedia Framework: G711 audio format definition [![Hex.pm](https://img.shields.io/hexpm/v/membrane_g711_format.svg)](https://hex.pm/api/packages/membrane_g711_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_g711_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_g711_format) - + +## membrane_av1_format +About Membrane Multimedia Framework: AV1 video format definition + +[![Hex.pm](https://img.shields.io/hexpm/v/membrane_av1_format.svg)](https://hex.pm/api/packages/membrane_av1_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_av1_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_av1_format) + ## membrane_h265_format -[Maintainer: [gBillal](https://github.com/gBillal)] H265 video format definition +[Maintainer: [gBillal](https://github.com/gBillal)] H265 video format definition [![Hex.pm](https://img.shields.io/hexpm/v/membrane_h265_format.svg)](https://hex.pm/api/packages/membrane_h265_format) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_h265_format/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/gBillal/membrane_h265_format) - + diff --git a/guides/packages/11_Standalone_media_libs.md b/guides/packages/12_Standalone_media_libs.md similarity index 93% rename from guides/packages/11_Standalone_media_libs.md rename to guides/packages/12_Standalone_media_libs.md index 2e94d3aec..1380b8bee 100644 --- a/guides/packages/11_Standalone_media_libs.md +++ b/guides/packages/12_Standalone_media_libs.md @@ -1,46 +1,46 @@ ## ex_webrtc -[Maintainer: [elixir-webrtc](https://github.com/elixir-webrtc)] An Elixir implementation of the W3C WebRTC API +[Maintainer: [elixir-webrtc](https://github.com/elixir-webrtc)] An Elixir implementation of the W3C WebRTC API [![Hex.pm](https://img.shields.io/hexpm/v/ex_webrtc.svg)](https://hex.pm/api/packages/ex_webrtc) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_webrtc/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/elixir-webrtc/ex_webrtc) - + ## ex_sdp -Parser and serializer for Session Description Protocol +Parser and serializer for Session Description Protocol [![Hex.pm](https://img.shields.io/hexpm/v/ex_sdp.svg)](https://hex.pm/api/packages/ex_sdp) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_sdp/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_sdp) - + ## ex_libnice -Libnice-based Interactive Connectivity Establishment (ICE) protocol support for Elixir +Libnice-based Interactive Connectivity Establishment (ICE) protocol support for Elixir [![Hex.pm](https://img.shields.io/hexpm/v/ex_libnice.svg)](https://hex.pm/api/packages/ex_libnice) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_libnice/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_libnice) - + ## ex_libsrtp -Elixir bindings for libsrtp +Elixir bindings for libsrtp [![Hex.pm](https://img.shields.io/hexpm/v/ex_libsrtp.svg)](https://hex.pm/api/packages/ex_libsrtp) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_libsrtp/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_libsrtp) - + ## ex_m3u8 -Elixir package for serializing and deserializing M3U8 manifests. +Elixir package for serializing and deserializing M3U8 manifests. [![Hex.pm](https://img.shields.io/hexpm/v/ex_m3u8.svg)](https://hex.pm/api/packages/ex_m3u8) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_m3u8/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_m3u8) - + ## ex_hls -An Elixir package for handling HLS streams +An Elixir package for handling HLS streams [![Hex.pm](https://img.shields.io/hexpm/v/ex_hls.svg)](https://hex.pm/api/packages/ex_hls) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_hls/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_hls) - + ## ex_libsrt -Elixir bindings to libsrt library exposing client and server APIs +Elixir bindings to libsrt library exposing client and server APIs [![Hex.pm](https://img.shields.io/hexpm/v/ex_libsrt.svg)](https://hex.pm/api/packages/ex_libsrt) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/ex_libsrt/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/ex_libsrt) - + ## membrane_rtsp -RTSP client for Elixir +RTSP client for Elixir [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtsp.svg)](https://hex.pm/api/packages/membrane_rtsp) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtsp/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_rtsp) - + ## membrane_ffmpeg_generator -[Labs] FFmpeg video and audio generator for tests, benchmarks and demos. +[Labs] FFmpeg video and audio generator for tests, benchmarks and demos. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ffmpeg_generator.svg)](https://hex.pm/api/packages/membrane_ffmpeg_generator) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ffmpeg_generator/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework-labs/membrane_ffmpeg_generator) - + diff --git a/guides/packages/12_Utils.md b/guides/packages/13_Utils.md similarity index 90% rename from guides/packages/12_Utils.md rename to guides/packages/13_Utils.md index 4e23e9d45..29fa8ca96 100644 --- a/guides/packages/12_Utils.md +++ b/guides/packages/13_Utils.md @@ -1,56 +1,56 @@ ## unifex -Tool for generating interfaces between native C code and Elixir +Tool for generating interfaces between native C code and Elixir [![Hex.pm](https://img.shields.io/hexpm/v/unifex.svg)](https://hex.pm/api/packages/unifex) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/unifex/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/unifex) - + ## bundlex -Multiplatform app bundler tool for Elixir +Multiplatform app bundler tool for Elixir [![Hex.pm](https://img.shields.io/hexpm/v/bundlex.svg)](https://hex.pm/api/packages/bundlex) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/bundlex/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/bundlex) - + ## beamchmark -Elixir tool for benchmarking EVM performance +Elixir tool for benchmarking EVM performance [![Hex.pm](https://img.shields.io/hexpm/v/beamchmark.svg)](https://hex.pm/api/packages/beamchmark) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/beamchmark/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/beamchmark) - + ## bunch -A bunch of helper functions, intended to make life easier +A bunch of helper functions, intended to make life easier [![Hex.pm](https://img.shields.io/hexpm/v/bunch.svg)](https://hex.pm/api/packages/bunch) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/bunch/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/bunch) - + ## bunch_native -Native part of the Bunch package +Native part of the Bunch package [![Hex.pm](https://img.shields.io/hexpm/v/bunch_native.svg)](https://hex.pm/api/packages/bunch_native) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/bunch_native/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/bunch_native) - + ## shmex -Elixir bindings for shared memory +Elixir bindings for shared memory [![Hex.pm](https://img.shields.io/hexpm/v/shmex.svg)](https://hex.pm/api/packages/shmex) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/shmex/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/shmex) - + ## membrane_timestamp_queue -Queue that aligns streams from multiple sources basing on timestamps +Queue that aligns streams from multiple sources basing on timestamps [![Hex.pm](https://img.shields.io/hexpm/v/membrane_timestamp_queue.svg)](https://hex.pm/api/packages/membrane_timestamp_queue) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_timestamp_queue/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_timestamp_queue) - + ## membrane_common_c -Membrane Multimedia Framework: Common C Routines +Membrane Multimedia Framework: Common C Routines [![Hex.pm](https://img.shields.io/hexpm/v/membrane_common_c.svg)](https://hex.pm/api/packages/membrane_common_c) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_common_c/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_common_c) - + ## membrane_telemetry_metrics -Membrane tool for generating metrics +Membrane tool for generating metrics [![Hex.pm](https://img.shields.io/hexpm/v/membrane_telemetry_metrics.svg)](https://hex.pm/api/packages/membrane_telemetry_metrics) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_telemetry_metrics/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_telemetry_metrics) - + ## membrane_opentelemetry -[Labs] Utilities for using OpenTelemetry with Membrane +[Labs] Utilities for using OpenTelemetry with Membrane [![Hex.pm](https://img.shields.io/hexpm/v/membrane_opentelemetry.svg)](https://hex.pm/api/packages/membrane_opentelemetry) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_opentelemetry/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework-labs/membrane_opentelemetry) - + ## membrane_precompiled_dependency_provider -Provides URLs for precompiled dependencies used by Membrane plugins. +Provides URLs for precompiled dependencies used by Membrane plugins. [![Hex.pm](https://img.shields.io/hexpm/v/membrane_precompiled_dependency_provider.svg)](https://hex.pm/api/packages/membrane_precompiled_dependency_provider) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_precompiled_dependency_provider/) [![GitHub](https://img.shields.io/badge/github-code-white.svg?logo=github)](https://github.com/membraneframework/membrane_precompiled_dependency_provider) - + diff --git a/mix.exs b/mix.exs index 1449c53ca..87d712097 100644 --- a/mix.exs +++ b/mix.exs @@ -78,7 +78,6 @@ defmodule Membrane.Mixfile do } ], extras: extras(), - formatters: ["html"], logo: "assets/logo.svg", source_ref: @source_ref, assets: %{ @@ -119,7 +118,11 @@ defmodule Membrane.Mixfile do defp extras do [ + {"skills/membrane-framework/SKILL.md", + [title: "Membrane Framework AI Skill", hidden: true]}, "README.md", + {"guides/llms/packages_list.md", + [title: "Packages in the Membrane ecosystem", hidden: true]}, "CHANGELOG.md", "CONTRIBUTING.md", Path.wildcard("guides/upgrading/*.md"), @@ -251,7 +254,7 @@ defmodule Membrane.Mixfile do {:bunch, "~> 1.6"}, {:ratio, "~> 3.0 or ~> 4.0"}, # Development - {:ex_doc, "~> 0.39", only: :dev, runtime: false}, + {:ex_doc, "~> 0.40", only: :dev, runtime: false}, {:makeup_diff, "~> 0.1", only: :dev, runtime: false}, {:dialyxir, "~> 1.1", only: :dev, runtime: false}, {:credo, "~> 1.7", only: :dev, runtime: false}, diff --git a/scripts/elixir/hex_packages.exs b/scripts/elixir/hex_packages.exs index 06a02b418..a5bcdb4af 100644 --- a/scripts/elixir/hex_packages.exs +++ b/scripts/elixir/hex_packages.exs @@ -15,6 +15,9 @@ :membrane_fake_plugin, :membrane_transcoder_plugin, :membrane_generator_plugin, + :membrane_live_framerate_converter_plugin, + :membrane_whisper_plugin, + :membrane_yolo_plugin, :membrane_webrtc_plugin, :membrane_rtmp_plugin, :membrane_http_adaptive_stream_plugin, @@ -49,6 +52,7 @@ :membrane_h264_ffmpeg_plugin, :membrane_vpx_plugin, :membrane_abr_transcoder_plugin, + :membrane_vk_video_plugin, :membrane_h265_ffmpeg_plugin, :membrane_raw_audio_parser_plugin, :membrane_portaudio_plugin, @@ -66,6 +70,7 @@ :membrane_overlay_plugin, :membrane_ffmpeg_swscale_plugin, :membrane_ffmpeg_video_filter_plugin, + :membrane_video_mixer_plugin, :membrane_agora_plugin, :membrane_element_gcloud_speech_to_text, :membrane_element_ibm_speech_to_text, @@ -84,6 +89,7 @@ :membrane_vp8_format, :membrane_vp9_format, :membrane_g711_format, + :membrane_av1_format, :membrane_h265_format, :ex_webrtc, :ex_sdp, diff --git a/scripts/elixir/packages.exs b/scripts/elixir/packages.exs index 1e25f8e32..9304da71a 100644 --- a/scripts/elixir/packages.exs +++ b/scripts/elixir/packages.exs @@ -23,6 +23,9 @@ "membrane_generator_plugin", "kim-company/membrane_live_framerate_converter_plugin", "membrane_template_plugin", + {:subsection, "AI"}, + "membrane_whisper_plugin", + "membrane_yolo_plugin", {:subsection, "Streaming protocols"}, "membrane_webrtc_plugin", "membrane_rtmp_plugin", @@ -62,6 +65,7 @@ "membrane_h264_ffmpeg_plugin", "membrane_vpx_plugin", "membrane_abr_transcoder_plugin", + "membrane_vk_video_plugin", "gBillal/membrane_h265_ffmpeg_plugin", "binarynoggin/elixir-turbojpeg", "kim-company/membrane_subtitle_mixer_plugin", @@ -83,12 +87,10 @@ "membrane_overlay_plugin", "membrane_ffmpeg_swscale_plugin", "membrane_ffmpeg_video_filter_plugin", - "membrane_yolo_plugin", "kim-company/membrane_video_mixer_plugin", {:subsection, "External APIs"}, "membrane_aws_plugin", "membrane_agora_plugin", - "membrane_webrtc_live", "membrane_element_gcloud_speech_to_text", "membrane_element_ibm_speech_to_text", "YuzuTen/membrane_s3_plugin", @@ -108,6 +110,7 @@ "membrane_vp8_format", "membrane_vp9_format", "membrane_g711_format", + "membrane_av1_format", "gBillal/membrane_h265_format", {:section, "Standalone media libs"}, "elixir-webrtc/ex_webrtc", diff --git a/scripts/elixir/update_packages_list.exs b/scripts/elixir/update_packages_list.exs index 69e805dfe..bf37d3441 100644 --- a/scripts/elixir/update_packages_list.exs +++ b/scripts/elixir/update_packages_list.exs @@ -71,6 +71,7 @@ packages_blacklist = [ "membrane_rtc_engine_timescaledb", "github_actions_test", "membrane_ice_plugin", + "membrane_webrtc_live", "membrane_rtc_engine" ] @@ -88,6 +89,7 @@ unless Enum.empty?(lacking_repos) do raise """ The following repositories aren't mentioned in the package list: #{Enum.map_join(lacking_repos, ",\n", & &1.name)} + Put them in scripts/elixir/packages.exs, or if they aren't packages, add them to the blacklist in this script. """ end @@ -172,16 +174,23 @@ File.write!(Path.join(__DIR__, "hex_packages.exs"), hex_packages_text) # generate packages list in markdown -header = """ - -| Package | Description | Links | -| --- | --- | --- | -""" - generated_code_comment = "" -packages_md = +generate_packages_md_fun = fn packages, links_column? -> + header = + if links_column? do + """ + | Package | Description | Links | + | --- | --- | --- | + """ + else + """ + | Package | Description | + | --- | --- | + """ + end + packages |> Enum.map_reduce( %{is_header_present: false}, @@ -193,9 +202,13 @@ packages_md = {"\n#### " <> name, %{acc | is_header_present: false}} %{type: :package} = package, acc -> + maybe_links_cell = + if links_column?, do: " #{package.hex_badge} #{package.hexdocs_badge} |", else: "" + package_info = """ #{if acc.is_header_present, do: "", else: header}\ - | [#{package.name}](#{package.url}) | #{package.owner_prefix}#{package.description} | #{package.hex_badge} #{package.hexdocs_badge} |\ + | [#{package.name}](#{package.url}) | #{package.owner_prefix}#{package.description} |\ + #{maybe_links_cell}\ """ {package_info, %{acc | is_header_present: true}} @@ -203,6 +216,9 @@ packages_md = ) |> elem(0) |> Enum.join("\n") +end + +packages_md = generate_packages_md_fun.(packages, true) packages_md = """ @@ -238,10 +254,10 @@ packages %{type: :package} = package, acc -> package_info = """ ## #{package.name} - #{package.owner_prefix}#{package.description} + #{package.owner_prefix}#{package.description} #{package.hex_badge} #{package.hexdocs_badge} #{package.github_badge} - + """ files = @@ -280,4 +296,10 @@ packages if file_content != "", do: File.write!(file_path, "#{generated_code_comment}\n#{file_content}") end) +# replace packages list for LLMs + +llms_packages_md = generate_packages_md_fun.(packages, false) +llms_packages_list_path = "guides/llms/packages_list.md" +File.write!(llms_packages_list_path, llms_packages_md) + IO.puts("Packages updated successfully.") diff --git a/skills/membrane-framework/SKILL.md b/skills/membrane-framework/SKILL.md new file mode 100644 index 000000000..dca32491c --- /dev/null +++ b/skills/membrane-framework/SKILL.md @@ -0,0 +1,245 @@ +--- +name: membrane-framework +description: Work with the Membrane multimedia streaming framework in Elixir. Use this skill whenever the user is building or debugging Membrane pipelines, writing custom Elements, Bins, or Filters, connecting pads, implementing callbacks, handling stream formats or EOS, or asking about Membrane architecture. Trigger on any mention of membrane_core, membrane, membrane framework, Membrane.Pipeline, Membrane.Sink, Membrane.Source, Membrane.Filter, Membrane.Endpoint, Membrane.Bin, Membrane.Pad, or multimedia streaming in an Elixir context — even if the user doesn't say "Membrane" explicitly but is clearly working on this codebase. +--- + +# Membrane Framework + +**Package**: `membrane_core` ~> 1.2 | **Docs**: https://hexdocs.pm/membrane_core/ | **Module index**: https://hexdocs.pm/membrane_core/llms.txt | **Demos**: https://github.com/membraneframework/membrane_demo | **All packages**: [packages_list.md](../../guides/llms/packages_list.md) + +## How to Approach Tasks + +- **New component** — before writing a new element, check [packages_list.md](../../guides/llms/packages_list.md) to see if it already exists in an existing plugin; if not, identify subtype (Source/Filter/Sink/Endpoint/Bin), define pads, implement required callbacks (`handle_buffer/4` for filters/sinks, `handle_demand/5` for manual-flow sources) +- **Generating boilerplate** — use `mix membrane.gen.filter MyApp.MyFilter`, `mix membrane.gen.source`, `mix membrane.gen.sink`, `mix membrane.gen.endpoint`, `mix membrane.gen.bin` instead of writing component skeletons by hand +- **Choosing element subtype** — prefer `Filter` for transformations (has sensible defaults for stream_format forwarding); use `Endpoint` only when output is unrelated to input (e.g. a UDP Endpoint); use `Source`/`Sink` for pure producers/consumers +- **Flow control** — default to `:auto` on all pads; only use `:manual` when you need fine-grained backpressure control; Almost the only use case of `:push` are output pads of Sources/Endpoints that cannot control when they produce data, e.g. UDP Source/Endpoint. +- **Pipeline topology** — use the ChildrenSpec DSL (`child/2`, `get_child/1`, `via_in/2`, `via_out/2`) (more info: [Membrane.ChildrenSpec](https://hexdocs.pm/membrane_core/Membrane.ChildrenSpec.md)) +- **Static vs dynamic topology** — return `spec:` from `handle_init/2` for static pipelines; return additional `spec:` actions from any callback (e.g. `handle_child_notification/4`) to grow the topology at runtime +- **Naming children** — use atoms (`:source`) for singletons, tuples (`{:decoder, track_id}`) for multi-instance children of the same type +- **Detecting pipeline completion** — implement `handle_element_end_of_stream/4` in the pipeline to know when a sink's input pad received EOS; then return `{[terminate: :normal], state}` (doesn't work if sink is a Membrane.Bin - then expect a custom message from the bin in `handle_child_notification` callback instead, if the bin sends it) +- **Dynamic tracks** (demuxers, variable inputs) — use the Dynamic Pads Pattern below +- **Crash isolation** — group children with `{spec, group: , crash_group_mode: :temporary}`; handle recovery in `handle_crash_group_down/3`; see [Crash Groups guide](https://hexdocs.pm/membrane_core/crash_groups.md) +- **Inserting debug probes** — add `child(:probe, %Membrane.Debug.Filter{handle_buffer: &IO.inspect(&1, label: :buffer)})` between any two elements to log buffers without changing pipeline logic. You can use different logging functions than `IO.inspect/2`. More info: [Membrane.Debug.Filter](https://hexdocs.pm/membrane_core/Membrane.Debug.Filter.md). +- **Linking children** - linked children pads accepted formats must have non-empty intersections +- **Debugging** — check pad `accepted_format` compatibility +- **Callback context** — every callback receives `ctx`; key fields: `ctx.children`, `ctx.pads`, `ctx.playback`; crash callbacks also have `ctx.crash_initiator`, `ctx.exit_reason`, `ctx.group_name`; see [Pipeline.CallbackContext](https://hexdocs.pm/membrane_core/Membrane.Pipeline.CallbackContext.md), [Bin.CallbackContext](https://hexdocs.pm/membrane_core/Membrane.Bin.CallbackContext.md), [Element.CallbackContext](https://hexdocs.pm/membrane_core/Membrane.Element.CallbackContext.md) +- **Logging** — utilize `Membrane.Logger` instead of `Logger` in Membrane components; it prepends component path and name to log messages. Requires `require Membrane.Logger` in the module before calling any logging functions. +- **Never modify code in `deps/`** +- **Use `mix hex.info ` when you need to check the newest version of a plugin** +- **Search for appropriate plugins in `README.md`, in `all-packages` section** +- **Check input and output pad definitions of elements in `deps/` (use `cat | grep def_input_pad` and `cat | grep def_output pad`) to make sure output pad's `accepted_stream_format` is compatible with `accepted_stream_format` of the input pad which it is linked to.** +- **If the `accepted_stream_format` doesn't match, search for an element which can act as an adapter** +- **When constructing Membrane Pipeline, lean towards using most powerful Membrane Components, which are [Boombox.Bin](https://hexdocs.pm/boombox/llms.txt) and [Membrane.Transcoder](https://hexdocs.pm/membrane_transcoder_plugin/llms.txt), instead of using many smaller plugins** + +--- + +## Common Pitfalls + +- **Modifying code in `deps/` directory** - never do that +- **Using `child/2` for an already-spawned child** — `child/2` always spawns a new process; use `get_child/1` to reference an existing one; duplicating a name raises an error +- **Linking static pads outside the spawning spec** — static pads must be linked in the same `spec` that spawns the component; linking them later raises a `LinkError` +- **Not wiring a dynamic bin pad in `handle_pad_added/3`** — a dynamic bin input pad must be connected to an internal child within 5 seconds or a `LinkError` is raised +- **Heavy work in `handle_init/2`** — `handle_init` is synchronous and blocks the parent; move file I/O, network connections, etc. to `handle_setup/2` +- **Producing data before `handle_playing/2`** — pads are not ready until `:playing`; don't send buffers from `handle_setup/2` +- **Using `:push` flow control carelessly** — whenever it is possible use `:auto` instead (eventually `:manual`). Source/Endpoint output pads are the exception. +- **Returning non-spec actions from `handle_init/2`** — we recommend to return only `:spec` action from this callback. + +--- + +## Architecture + +``` +Pipeline +├── Element (Source/Filter/Sink/Endpoint) ← leaf, processes data +├── Bin ← dual role: parent (has children) + child (has pads) +│ ├── Element +│ └── Bin ← bins nest arbitrarily deep +└── ... +``` + +| Type | Parent | Child | Has Pads | +|------|--------|-------|----------| +| **Pipeline** | yes | no | no | +| **Bin** | yes | yes | yes | +| **Element** | no | yes | yes | + +Element subtypes: **Source** (output only) · **Filter** (in + out, output is transformed input) · **Sink** (input only) · **Endpoint** (in + out, but output might be not related to input) + +--- + +## Pads + +Defined on Elements and Bins (not Pipelines) using `def_input_pad/2` (`Membrane.Element.WithInputPads`) and `def_output_pad/2` (`Membrane.Element.WithOutputPads`): + +```elixir +def_input_pad :input, accepted_format: _any +def_output_pad :output, accepted_format: Membrane.RawAudio, flow_control: :auto +``` + +- **Availability**: `:always` (static, one instance, referenced by atom) or `:on_request` (dynamic, reference via `Pad.ref(:name, id)`) +- **Flow control**: `:auto` (framework manages demand — preferred), `:manual` (explicit via `:demand`/`:redemand`), `:push` (no demand, risk of overflow) +- One input pad ↔ one output pad only; pads must have compatible `accepted_format` +- Default pad names `:input`/`:output` allow omitting `via_in`/`via_out` in specs +- **`accepted_format` matching syntax**: `_any` (accept anything) · `Membrane.RawAudio` (any struct of that type) · `%Membrane.RawAudio{channels: 2}` (match specific fields) · `%Membrane.RemoteStream{}` (unknown/unparsed stream). `any_of(patter1, pattern2, ...)` matches if any pattern matches. +- **Full pads guide** (static vs dynamic, bin pads, lifecycle): [Everything about pads](https://hexdocs.pm/membrane_core/pads.md) + +--- + +## Component Lifecycle + +``` +handle_init/2 sync, blocks parent — parse opts, return initial spec +handle_setup/2 async — heavy init (open files, connect services) + return {[setup: :incomplete], state} to delay :playing +handle_pad_added/3 fires for dynamic pads linked in the same spec +handle_playing/2 component is ready — start producing/consuming data +``` + +All components spawned in the same `:spec` action enter `:playing` together (they synchronize to the slowest setup). Elements and Bins wait for their parent before `handle_playing/2`. + +**Stream format and EOS rules (critical for filter authors):** +- A source/filter **must send `{:stream_format, {pad, format}}` before the first buffer** on each output pad, or downstream elements crash +- The default [`handle_stream_format/4`](https://hexdocs.pm/membrane_core/Membrane.Element.WithInputPads.md) in filters forwards the format downstream — if you override it, you must forward manually or return the `{:stream_format, ...}` action yourself +- The default [`handle_end_of_stream/3`](https://hexdocs.pm/membrane_core/Membrane.Element.WithInputPads.md) in filters forwards EOS downstream — overriding without forwarding will stall the pipeline + +Full lifecycle guide: [Lifecycle of Membrane Components](https://hexdocs.pm/membrane_core/components_lifecycle.md) + +--- + +## Pipeline & Bin DSL (ChildrenSpec) + +```elixir +# Linear chain — child/2 spawns a new named child +child(:source, %Membrane.File.Source{location: "input.mp4"}) +|> child(:filter, MyFilter) +|> child(:sink, %Membrane.File.Sink{location: "out.raw"}) + +# Explicit pad names (required for non-default names or dynamic pads) +get_child(:demuxer) +|> via_out(Pad.ref(:output, track_id)) +|> via_in(:video_input) +|> child(:decoder, Membrane.H264.FFmpeg.Decoder) + +# Link to an already-existing child +get_child(:existing_filter) |> child(:new_sink, MySink) + +# Inside a Bin — bin_input/bin_output connect the bin's own pads to internal children +bin_input(:input) |> child(:filter, MyFilter) |> bin_output(:output) + +# Crash group — all children in the spec share the group; a crash in any terminates all +{child(:source, Source) |> child(:sink, Sink), group: :my_group, crash_group_mode: :temporary} +``` + +**Bin pad wiring rules:** +- `bin_input(pad_ref)` / `bin_output(pad_ref)` are the interior side of the bin's own pads +- Dynamic bin input pads **must** be wired inside `handle_pad_added/3` within 5 seconds or a `LinkError` is raised +- Linking to a bin actually links directly to the inner component (no extra message hop) + +--- + +## Dynamic Pads Pattern + +The standard approach for variable-track streams (e.g. MP4 demuxers): + +```elixir +# 1. Spawn source + demuxer; demuxer hasn't identified tracks yet +def handle_init(_ctx, state) do + {[spec: child(:source, Source) |> child(:demuxer, Demuxer)], state} +end + +# 2. Demuxer notifies parent once tracks are known +def handle_child_notification({:new_tracks, tracks}, :demuxer, _ctx, state) do + spec = Enum.map(tracks, fn {id, _fmt} -> + get_child(:demuxer) + |> via_out(Pad.ref(:output, id)) + |> child({:decoder, id}, Decoder) + |> child({:sink, id}, Sink) + end) + {[spec: spec], state} +end +``` + +--- + +## Built-in Utility Elements + +| Module | Purpose | +|--------|---------| +| `Membrane.Funnel` | Multiple inputs → one output | +| `Membrane.Tee` | One input → multiple outputs | +| `Membrane.Connector` | Connect dynamic pads with internal buffering | +| `Membrane.Testing.Source` | Inject buffers into a pipeline in tests | +| `Membrane.Testing.Sink` | Capture and assert on buffers in tests | +| `Membrane.Debug.Filter` | Log/inspect buffers flowing through pipeline | +| `Membrane.Debug.Sink` | Log/inspect buffers at pipeline end | +| `Membrane.FilterAggregator` | It is deprecated, just don't use it | + +--- + +## Testing + +```elixir +import Membrane.ChildrenSpec +import Membrane.Testing.Assertions +alias Membrane.Testing + +pipeline = Testing.Pipeline.start_link_supervised!(spec: [ + child(:source, %Testing.Source{output: [<<1, 2, 3>>, <<4, 5, 6>>]}) + |> child(:sink, Testing.Sink) +]) + +assert_sink_buffer(pipeline, :sink, %Membrane.Buffer{payload: <<1, 2, 3>>}) +assert_start_of_stream(pipeline, :sink) +assert_end_of_stream(pipeline, :sink) +``` + +- `Testing.DynamicSource` — like `Testing.Source` but with a dynamic output pad + +--- + +## Timing + +All timestamps are `Membrane.Time.t()`. It is integer nanoseconds under the hood, but don't use this information, because it is a part of the private API. However, keep in mind you can perform operations on `Membrane.Time.t()` using `+` or `-` operators. Helpers: `Membrane.Time.seconds/1`, `Membrane.Time.milliseconds/1`, `Membrane.Time.microseconds/1`, etc. Timers started with `:start_timer` action fire `handle_tick/3`. + +More info: [Membrane.Time](https://hexdocs.pm/membrane_core/Membrane.Time.md), [Timestamps guide](https://hexdocs.pm/membrane_core/timestamps.md). + +--- + +## Actions + +Actions are returned from callbacks as `{[action_list], state}`. Full reference by component type: + +- [Membrane.Element.Action](https://hexdocs.pm/membrane_core/Membrane.Element.Action.md) — buffers, stream_format, events, EOS, demand, timers, notify_parent, setup, terminate +- [Membrane.Bin.Action](https://hexdocs.pm/membrane_core/Membrane.Bin.Action.md) — spec, remove_children, remove_link, notify_parent, notify_child, timers, setup, terminate +- [Membrane.Pipeline.Action](https://hexdocs.pm/membrane_core/Membrane.Pipeline.Action.md) — spec, remove_children, remove_link, notify_child, timers, terminate + +--- + +## Key Source Files + +| Module | Purpose | +|--------|---------| +| [`Membrane.Pipeline`](https://hexdocs.pm/membrane_core/Membrane.Pipeline.md) | Pipeline behaviour & all callbacks | +| [`Membrane.Pipeline.Action`](https://hexdocs.pm/membrane_core/Membrane.Pipeline.Action.md) | Pipeline action type specs | +| [`Membrane.Bin`](https://hexdocs.pm/membrane_core/Membrane.Bin.md) | Bin behaviour & all callbacks | +| [`Membrane.Bin.Action`](https://hexdocs.pm/membrane_core/Membrane.Bin.Action.md) | Bin action type specs | +| [`Membrane.Element.Base`](https://hexdocs.pm/membrane_core/Membrane.Element.Base.md) | Shared element callbacks | +| [`Membrane.Element.WithInputPads`](https://hexdocs.pm/membrane_core/Membrane.Element.WithInputPads.md) | `handle_buffer/4`, `handle_stream_format/4`, `handle_end_of_stream/3` | +| [`Membrane.Element.WithOutputPads`](https://hexdocs.pm/membrane_core/Membrane.Element.WithOutputPads.md) | `handle_demand/5` | +| [`Membrane.Element.Action`](https://hexdocs.pm/membrane_core/Membrane.Element.Action.md) | Element action type specs | +| [`Membrane.Pad`](https://hexdocs.pm/membrane_core/Membrane.Pad.md) | Pad definitions, `Pad.ref/2` | +| [`Membrane.Buffer`](https://hexdocs.pm/membrane_core/Membrane.Buffer.md) | Buffer struct | +| [`Membrane.ChildrenSpec`](https://hexdocs.pm/membrane_core/Membrane.ChildrenSpec.md) | Topology DSL | + +--- + +## Callback Reference + +Callbacks are documented in the relevant behaviour modules: + +- [Membrane.Pipeline](https://hexdocs.pm/membrane_core/Membrane.Pipeline.md) — `handle_init`, `handle_setup`, `handle_playing`, `handle_call`, `handle_child_notification`, `handle_child_terminated`, `handle_crash_group_down`, `handle_element_end_of_stream`, etc. +- [Membrane.Bin](https://hexdocs.pm/membrane_core/Membrane.Bin.md) — same as Pipeline plus `handle_pad_added`, `handle_pad_removed`, `handle_parent_notification` +- [Membrane.Element.Base](https://hexdocs.pm/membrane_core/Membrane.Element.Base.md) — callbacks common to all elements: `handle_init`, `handle_setup`, `handle_playing`, `handle_pad_added`, `handle_pad_removed`, `handle_parent_notification`, `handle_info`, `handle_tick` +- [Membrane.Element.WithInputPads](https://hexdocs.pm/membrane_core/Membrane.Element.WithInputPads.md) — `handle_buffer`, `handle_stream_format`, `handle_start_of_stream`, `handle_end_of_stream` +- [Membrane.Element.WithOutputPads](https://hexdocs.pm/membrane_core/Membrane.Element.WithOutputPads.md) — `handle_demand` +- [Membrane.Source](https://hexdocs.pm/membrane_core/Membrane.Source.md), [Membrane.Filter](https://hexdocs.pm/membrane_core/Membrane.Filter.md), [Membrane.Sink](https://hexdocs.pm/membrane_core/Membrane.Sink.md), [Membrane.Endpoint](https://hexdocs.pm/membrane_core/Membrane.Endpoint.md) — combine the above with default implementations