Skip to content

Timestamps guide - #1033

Merged
Noarkhh merged 9 commits into
masterfrom
timestamps-guide
Jan 14, 2026
Merged

Timestamps guide#1033
Noarkhh merged 9 commits into
masterfrom
timestamps-guide

Conversation

@Noarkhh

@Noarkhh Noarkhh commented Dec 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

No description provided.

@Noarkhh Noarkhh self-assigned this Dec 3, 2025
@Noarkhh Noarkhh added this to Smackore Dec 3, 2025
@Noarkhh Noarkhh moved this to In Progress in Smackore Dec 3, 2025
@Noarkhh Noarkhh added the no-changelog This label has to be added if changes from the PR are not meant to be placed in the CHANGELOG.md label Dec 3, 2025
@Noarkhh
Noarkhh changed the base branch from master to dynamic-pads-guide December 3, 2025 18:12
@Noarkhh Noarkhh linked an issue Dec 4, 2025 that may be closed by this pull request
Identify and add missing explanations for Membrane concepts and use cases #1022
Closed
@Noarkhh
Noarkhh force-pushed the dynamic-pads-guide branch 2 times, most recently from 9841fae to b4539b0 Compare December 19, 2025 12:46
Base automatically changed from dynamic-pads-guide to master December 19, 2025 12:51
@Noarkhh
Noarkhh marked this pull request as ready for review January 7, 2026 10:59
@Noarkhh
Noarkhh requested a review from mat-hek as a code owner January 7, 2026 10:59
@Noarkhh Noarkhh moved this from In Progress to In Review in Smackore Jan 7, 2026
@Noarkhh
Noarkhh requested review from FelonEkonom and varsill January 7, 2026 10:59
mat-hek
mat-hek reviewed Jan 7, 2026
View reviewed changes

@mat-hek mat-hek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great. I'm missing some summary/guidelines, that would emphasise that:

  • DTS should always be monotonic, while PTS for video with B-frames can be non-monotonic
  • Filters should always forward timestamps
  • If a filter doesn't use timestamps, it should forward them, but not rely on them being set
  • Sources should attach timestamps to buffers whenever they're known
  • Whenever possible, elements should rely on timestamps instead of framerate or audio duration calculated from the stream
  • If an element queues buffers in its state (or uses a library that does so), it should make sure that the timestamps for the output buffers are the same as for the corresponding input buffers
  • You should make sure that calculations on timestamps don't introduce an accumulating error. Prefer using rationals (Ratio library) to floats.
  • Elements should generate deterministic output timestamps for better testability
  • If an element transforms N input buffers into M output buffers, each of the output buffers should have either:
    • the timestamp of the first of the input buffers (even if only a part of it was used to construct the output buffers)
    • more precise timestamps, if it's possible to calculate them
  • Timestamps are harder than they seem and are the source of many bugs, including:
    • Audio/video desynchronization
    • Stream hanging (due to waiting indefinitely to process/play a buffer because of a wrong timestamp)
    • Stream stalls (e.g. due to processing a real-time stream slightly faster than real time)
    • Memory leaks (e.g. due to processing a real-time stream slightly slower than real time and indefinite buffering)
    • Video flickering (due to incorrect handling of B-frames)
    • Audio cracking
      Therefore, the operations on timestamps should be given a lot of care and be well-tested

Comment thread guides/useful_concepts/timestamps.md Outdated
@@ -0,0 +1,100 @@
# Timestamps

In a nutshell, timestamps determine when a given event occurred in time. For

@mat-hek mat-hek Jan 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In a nutshell, timestamps determine when a given event occurred in time. For
In a nutshell, timestamps determine when a given event occurred (or should occur) in time. For

Comment thread guides/useful_concepts/timestamps.md Outdated
send them over as fast as possible, which is not something we want. We want the receiver
to get the stream in realtime, so that they can display it as it comes.

## Decoding Time Stamps

@mat-hek mat-hek Jan 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Decoding Time Stamps
## Decode Time Stamps (DTS)

@Noarkhh
Noarkhh requested a review from mat-hek January 8, 2026 16:42
FelonEkonom
FelonEkonom approved these changes Jan 8, 2026
View reviewed changes
Comment thread guides/useful_concepts/timestamps.md Outdated
[Realtimer](https://hexdocs.pm/membrane_realtimer_plugin/Membrane.Realtimer.html)
is an element from
[membrane_realtimer_plugin](https://hex.pm/packages/membrane_realtimer_plugin).
It takes in a stream and limits its flow according to its PTSs. For example, if

@FelonEkonom FelonEkonom Jan 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, it will take dts || pts

Comment thread guides/useful_concepts/timestamps.md Outdated
should make sure that the timestamps for the output buffers are the same as for
the corresponding input buffers.
* You should ensure that calculations on timestamps don't introduce an
accumulating error. Prefer using rationals (Ratio library) to floats.

@FelonEkonom FelonEkonom Jan 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe some link to ratio?

@mat-hek mat-hek Jan 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could also state 'prefer using integers or rationals' (yes I know who wrote this :P)

Comment thread guides/useful_concepts/timestamps.md Outdated
to create a term representing three seconds, we call `Membrane.Time.seconds(3)`.
* To read the amount of time represented, we can use `Membrane.Time.as_<unit>/2`
functions. For example, to get an amount of milliseconds represented by a time,
we call `Membrane.Time.as_milliseconds(some_time)`

@FelonEkonom FelonEkonom Jan 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Membrane.Time.as_milliseconds/1? It would generate a link to as_milliseconds docs, I guess

varsill
varsill requested changes Jan 13, 2026
View reviewed changes
Comment thread guides/useful_concepts/timestamps.md
Comment thread guides/useful_concepts/timestamps.md Outdated
Comment thread guides/useful_concepts/timestamps.md
Comment thread guides/useful_concepts/timestamps.md Outdated
Comment thread guides/useful_concepts/timestamps.md Outdated
Comment thread guides/useful_concepts/timestamps.md Outdated
Comment on lines +110 to +111
* If a filter doesn't use timestamps, it should forward them, but not rely on
them being set.

@varsill varsill Jan 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little bit vague to me, how could a filter "rely on" timestamps being set and not use them?

@mat-hek mat-hek Jan 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my idea, and indeed it doesn't make much sense :P

Suggested change
* If a filter doesn't use timestamps, it should forward them, but not rely on
them being set.
* If a filter doesn't use timestamps, it should still forward them.
* If an element relies on timestamps and they are not set, it should raise a meaningful error.

FelonEkonom
FelonEkonom approved these changes Jan 13, 2026
View reviewed changes

@FelonEkonom FelonEkonom left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:feelsgood:

varsill
varsill approved these changes Jan 14, 2026
View reviewed changes

@varsill varsill left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇

Comment thread guides/useful_concepts/timestamps.md Outdated
Comment on lines +28 to +29
to round the result to the nearest integer or `:exact` mode to get the result
as a [rational number](https://hexdocs.pm/ratio/Ratio.html#t:t/0).

@varsill varsill Jan 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can mention that :exact is a default value

@Noarkhh
Noarkhh merged commit 1caffa3 into master Jan 14, 2026
5 of 6 checks passed
@Noarkhh
Noarkhh deleted the timestamps-guide branch January 14, 2026 16:15
@github-project-automation github-project-automation Bot moved this from In Review to Done in Smackore Jan 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@varsill varsill varsill approved these changes
@FelonEkonom FelonEkonom FelonEkonom approved these changes
@mat-hek mat-hek Awaiting requested review from mat-hek mat-hek is a code owner

Assignees

@Noarkhh Noarkhh

Labels

no-changelog This label has to be added if changes from the PR are not meant to be placed in the CHANGELOG.md

Projects

Smackore
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Identify and add missing explanations for Membrane concepts and use cases

4 participants

@Noarkhh @varsill @mat-hek @FelonEkonom