Merge pull request #13 from mondain/jemalloc

Author: mondain

README.md

@@ -12,7 +12,7 @@ It turns MP4 input into CMSF-style publishable objects, builds draft-aware MOQT
 - Remuxes progressive MP4 input into synthesized fragmented media objects.
 - Extracts track metadata and RFC 6381 codec identifiers.
 - Preserves HEVC signaling and normalizes `hev1` to `hvc1` when needed.
-- Builds publish plans with catalog, optional SAP event timeline metadata, and media objects.
+- Builds publish plans with catalog, optional MSF media timeline and SAP event timeline metadata, and media objects.
 - Emits generated objects and catalog metadata to disk for inspection.
 - Supports draft-aware MOQT framing for drafts 14, 16, and 18.
 - Publishes over Raw QUIC or WebTransport when picoquic and picotls are available.

docs/design.md

@@ -27,11 +27,20 @@ The catalog format includes:
 - `role` with values such as `video` and `audio`
 - RFC 6381 `codec` strings such as `avc1.64000C`, `mp4a.40.2`, and HEVC values like `hvc1.1.6.L90.B0`
 - `renderGroup` and `isLive`
+- MSF media timeline tracks with `packaging: "mediatimeline"` and `role: "mediatimeline"`
 - SAP event timeline tracks with `packaging: "eventtimeline"` and `eventType: "org.ietf.moq.cmsf.sap"`
 - `width` and `height` for video tracks
 - `sampleRate` and `channelCount` for audio tracks
 - base64-encoded per-track CMAF initialization segment (`ftyp` + `moov`) in `initData`
 
+With `--msf-timeline`, `catalog.json` also includes a `timeline` metadata track with:
+
+- `packaging: "mediatimeline"`
+- `mimeType: "application/json"`
+- `depends` pointing to all media tracks
+
+The corresponding `timeline_g0_o0.json` payload is an explicit MSF media timeline array of `[mediaTimeMs, [groupId, objectId], wallclockMs]` records. VOD output uses `0` for wallclock time.
+
 With `--sap`, `catalog.json` also includes:
 
 - one `*_sap` track per media track

docs/publisher-api.md

@@ -27,6 +27,7 @@ config.track_namespace = "media";
 config.forward = false;
 config.publish_catalog = false;
 config.include_sap = false;
+config.include_msf_timeline = false;
 config.split_cmaf_chunks = true;
 config.paced = false;
 config.loop = false;
@@ -134,7 +135,8 @@ Convenience helpers:
 
 ## 7. Live Input Publish (Incremental stdin/stream)
 
-For live pipelines (for example ffmpeg piping fragmented MP4):
+The default live path expects fragmented MP4, which matches ffmpeg/CMAF
+pipelines:
 
 ```cpp
 const auto status = publisher.publish_live(std::cin, endpoint, tls);
@@ -150,7 +152,39 @@ if (!status.ok) {
 
 `publish_live(...)` uses incremental parsing and live publish flow instead of buffering to EOF.
 
-## 8. ALPN Override Behavior
+## 8. Arbitrary Live Object Publish
+
+Applications that already produce MoQ objects directly can bypass fragmented MP4
+ingest with `publish_live_objects(...)`.
+
+```cpp
+std::vector<openmoq::publisher::LiveObject> objects = {
+    {
+        .track_name = "events",
+        .group_id = 0,
+        .object_id = 0,
+        .payload = {'h', 'e', 'l', 'l', 'o'},
+    },
+};
+std::size_t next = 0;
+
+openmoq::publisher::LiveObjectSource source;
+source.tracks = {{.track_name = "events"}};
+source.next_object = [&]() -> std::optional<openmoq::publisher::LiveObject> {
+    if (next >= objects.size()) {
+        return std::nullopt;
+    }
+    return objects[next++];
+};
+
+const auto status = publisher.publish_live_objects(source, endpoint, tls);
+```
+
+Each `LiveObject` supplies the target track, group/object IDs, optional media
+timing, and the payload bytes to send. The fragmented MP4 `publish_live(...)`
+API remains the default live publishing path.
+
+## 9. ALPN Override Behavior
 
 By default, the API applies transport-appropriate ALPN:
 
@@ -177,8 +211,9 @@ The same override flag exists on:
 - `publish_file(...)`
 - `publish_stream(...)`
 - `publish_live(...)`
+- `publish_live_objects(...)`
 
-## 9. Error Handling Pattern
+## 10. Error Handling Pattern
 
 All API publish calls return `TransportStatus`:
 
@@ -200,18 +235,19 @@ if (!status.ok) {
 }
 ```
 
-## 10. Integration Pattern for Larger Applications
+## 11. Integration Pattern for Larger Applications
 
 For service-style integration:
 
 1. Construct one `Publisher` per runtime configuration profile.
 2. On ingest, call `prepare_file(...)` or `prepare_stream(...)`.
 3. Store or inspect `PreparedPublish` metadata as needed.
 4. Publish to one or more endpoints with `publish(...)`.
-5. For continuous input, run `publish_live(...)` in a worker thread.
-6. Use `TransportStatus` messages for metrics and retry decisions.
+5. For continuous fragmented MP4 input, run `publish_live(...)` in a worker thread.
+6. For direct object producers, provide a `LiveObjectSource` and call `publish_live_objects(...)`.
+7. Use `TransportStatus` messages for metrics and retry decisions.
 
-## 11. Publish Summary (`stats`)
+## 12. Publish Summary (`stats`)
 
 The publisher API is blocking: `publish(...)`, `publish_file(...)`,
 `publish_stream(...)`, and `publish_live(...)` run the session on the calling
@@ -243,6 +279,7 @@ Current fields:
 - `groupsPublished`: total (track, group) units published in the current/last session
 - `splitCmafChunks`: current packaging mode (`true` = split chunks, `false` = coalesced chunks)
 - `includeSap`: whether SAP track/object packaging is enabled
+- `includeMsfTimeline`: whether MSF media timeline track/object packaging is enabled
 - `transport`, `host`, `port`, `path`: endpoint context for the current/last session
 - `connectionId`: last known transport connection ID
 - `lastError`: last publisher-level error, if any
@@ -269,6 +306,7 @@ Example:
   "groupsPublished": 42,
   "splitCmafChunks": true,
   "includeSap": false,
+  "includeMsfTimeline": false,
   "transport": "webtransport",
   "host": "relay.example.com",
   "port": 443,

docs/quickstart.md

@@ -16,6 +16,12 @@ Inspect the same input with SAP event timeline metadata enabled:
 ./build/openmoq-publisher --input sample.mp4 --sap --dump-plan
 ```
 
+Inspect the same input with an MSF media timeline track enabled:
+
+```bash
+./build/openmoq-publisher --input sample.mp4 --msf-timeline --dump-plan
+```
+
 Try the draft-16 compatibility profile:
 
 ```bash
@@ -42,7 +48,7 @@ The output directory should contain:
 - one `*_probe.mp4` file per emitted media object for direct `ffprobe` use
 - `publish-plan.txt`
 
-When `--sap` is enabled, the output directory also contains one `*_sap_g*_o*.json` file per emitted SAP event timeline object.
+When `--msf-timeline` is enabled, the output directory also contains `timeline_g0_o0.json` with explicit MSF media timeline records. When `--sap` is enabled, the output directory also contains one `*_sap_g*_o*.json` file per emitted SAP event timeline object.
 
 ## Use Standard Input
 
@@ -104,6 +110,7 @@ cat sample.mp4 | ./build/openmoq-publisher \
 ## Output Notes
 
 - default output includes the `catalog` object plus media objects
+- `--msf-timeline` additionally creates a `timeline` media timeline track and metadata object
 - `--sap` additionally creates `*_sap` metadata tracks and objects
 - default packaging emits lower-latency split MOQT objects per group when chunk/sample boundaries are available
 - `--coalesce-cmaf-chunks` restores one media object per group

docs/testing.md

@@ -26,6 +26,7 @@ Publish-plan numbering notes:
 - `group_id` is allocated per track, not across all tracks, so interleaved audio and video fragments can both use `0, 1, 2, ...`
 - by default, `object_id` advances within a group when CMAF content is split into multiple MOQT objects for lower latency
 - `--coalesce-cmaf-chunks` forces `object_id = 0` for the current one-object-per-group fallback
+- MSF media timeline tracks are disabled by default; add `--msf-timeline` when you want a `timeline` metadata track and object
 - SAP event timeline tracks are disabled by default; add `--sap` when you want catalog and metadata objects for `*_sap` tracks
 
 If you want the packaging and transport tests without the CLI target, use the secondary build tree:
@@ -61,6 +62,7 @@ Input source notes:
 - `--input <path>` reads an MP4 from a regular file
 - `--input -` reads the MP4 byte stream from standard input
 - fragmented and progressive MP4 inputs are both supported through either source type
+- MSF media timeline tracks are disabled by default; add `--msf-timeline` to include the `timeline` metadata track and object in the publish plan
 - SAP event timeline tracks are disabled by default; add `--sap` to include `*_sap` metadata tracks and objects in the publish plan
 
 Use `--dump-plan` to inspect the generated publish plan without touching the network:
@@ -75,6 +77,12 @@ Include SAP event timeline output explicitly:
 ./build/openmoq-publisher --input sample.mp4 --draft 14 --sap --dump-plan
 ```
 
+Include MSF media timeline output explicitly:
+
+```bash
+./build/openmoq-publisher --input sample.mp4 --draft 14 --msf-timeline --dump-plan
+```
+
 Use stdin when the source is already being produced by another command:
 
 ```bash

include/openmoq/publisher/cli_options.h

@@ -32,6 +32,7 @@ struct CliOptions {
     bool forward = false;
     bool publish_catalog = false;
     bool include_sap = false;
+    bool include_msf_timeline = false;
     bool split_cmaf_chunks = true;
     bool paced = false;
     bool loop = false;

include/openmoq/publisher/cmsf_packager.h

@@ -41,7 +41,10 @@ struct PublishPlan {
     std::vector<CmsfObject> objects;
 };
 
-PublishPlan build_publish_plan(const SegmentedMp4& segmented_mp4, DraftVersion version, bool include_sap = false);
+PublishPlan build_publish_plan(const SegmentedMp4& segmented_mp4,
+                               DraftVersion version,
+                               bool include_sap = false,
+                               bool include_msf_timeline = false);
 std::string render_publish_plan(const PublishPlan& plan);
 PublishPlan materialize_publish_plan(const PublishPlan& plan, std::span<const std::uint8_t> bytes);
 void emit_plan_objects(const PublishPlan& plan,

include/openmoq/publisher/live_object.h

@@ -0,0 +1,32 @@
+#pragma once
+
+#include <cstdint>
+#include <functional>
+#include <optional>
+#include <string>
+#include <vector>
+
+namespace openmoq::publisher {
+
+struct LiveTrack {
+    std::string track_name;
+};
+
+struct LiveObject {
+    std::string track_name;
+    std::size_t group_id = 0;
+    std::uint64_t subgroup_id = 0;
+    std::size_t object_id = 0;
+    std::uint64_t media_time_us = 0;
+    std::uint64_t media_duration_us = 0;
+    std::vector<std::uint8_t> payload;
+    bool subgroup_contains_group_largest = true;
+    bool final_in_subgroup = true;
+};
+
+struct LiveObjectSource {
+    std::vector<LiveTrack> tracks;
+    std::function<std::optional<LiveObject>()> next_object;
+};
+
+}  // namespace openmoq::publisher

include/openmoq/publisher/publisher_api.h

@@ -13,6 +13,7 @@
 #include <vector>
 
 #include "openmoq/publisher/cmsf_packager.h"
+#include "openmoq/publisher/live_object.h"
 #include "openmoq/publisher/moq_draft.h"
 #include "openmoq/publisher/transport/publisher_transport.h"
 
@@ -24,6 +25,7 @@ struct PublisherConfig {
     bool forward = false;
     bool publish_catalog = false;
     bool include_sap = false;
+    bool include_msf_timeline = false;
     bool split_cmaf_chunks = true;
     bool paced = false;
     bool loop = false;
@@ -42,6 +44,7 @@ struct PublisherStats {
     std::uint64_t groups_published = 0;
     bool split_cmaf_chunks = true;
     bool include_sap = false;
+    bool include_msf_timeline = false;
     transport::TransportKind transport = transport::TransportKind::kRawQuic;
     std::string host;
     std::uint16_t port = 0;
@@ -82,6 +85,10 @@ class Publisher {
                                             const transport::EndpointConfig& endpoint,
                                             const transport::TlsConfig& tls = {},
                                             bool endpoint_alpn_overridden = false) const;
+    transport::TransportStatus publish_live_objects(const LiveObjectSource& source,
+                                                    const transport::EndpointConfig& endpoint,
+                                                    const transport::TlsConfig& tls = {},
+                                                    bool endpoint_alpn_overridden = false) const;
     transport::TransportStatus disconnect(std::uint64_t application_error_code = 0) const;
     PublisherStats stats() const;
     [[deprecated("Use stats(); live polling is not supported by the blocking publish API.")]]

include/openmoq/publisher/transport/moqt_session.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include "openmoq/publisher/cmsf_packager.h"
+#include "openmoq/publisher/live_object.h"
 #include "openmoq/publisher/transport/publisher_transport.h"
 
 #include <iosfwd>
@@ -42,6 +43,8 @@ class MoqtSession {
     TransportStatus publish_live(std::istream& input,
                                  openmoq::publisher::DraftVersion draft_version,
                                  bool split_cmaf_chunks);
+    TransportStatus publish_live_objects(const openmoq::publisher::LiveObjectSource& source,
+                                         openmoq::publisher::DraftVersion draft_version);
     TransportStatus close(std::uint64_t application_error_code = 0);
     PublishStats publish_stats() const;