Refresh managed dependencies daily

mondain · mondain · commit 3a9ac9cd4abc · 2026-08-04T14:49:30.000-07:00
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -29,29 +29,6 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v5
 
-      - name: Checkout picoquic
-        uses: actions/checkout@v5
-        with:
-          repository: private-octopus/picoquic
-          ref: master
-          path: third_party/picoquic
-
-      - name: Checkout picotls
-        uses: actions/checkout@v5
-        with:
-          repository: private-octopus/picotls
-          ref: master
-          path: third_party/picotls
-          submodules: recursive
-
-      - name: Checkout moq5
-        if: matrix.backend == 'libmoq'
-        uses: actions/checkout@v5
-        with:
-          repository: openmoq/moq5
-          ref: main
-          path: third_party/moq5
-
       - name: Show toolchain
         shell: bash
         run: |
@@ -98,8 +75,6 @@ jobs:
           args=(
             cmake -S . -B build
             -DCMAKE_POLICY_VERSION_MINIMUM=3.5
-            "-DOPENMOQ_PICOQUIC_SOURCE_DIR=${{ github.workspace }}/third_party/picoquic"
-            "-DOPENMOQ_PICOTLS_SOURCE_DIR=${{ github.workspace }}/third_party/picotls"
           )
           if [ "${{ matrix.backend }}" = "libmoq" ]; then
             args+=(
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -31,21 +31,6 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v5
 
-      - name: Checkout picoquic
-        uses: actions/checkout@v5
-        with:
-          repository: private-octopus/picoquic
-          ref: master
-          path: third_party/picoquic
-
-      - name: Checkout picotls
-        uses: actions/checkout@v5
-        with:
-          repository: private-octopus/picotls
-          ref: master
-          path: third_party/picotls
-          submodules: recursive
-
       - name: Set up Windows dependencies
         if: runner.os == 'Windows'
         shell: pwsh
@@ -86,8 +71,6 @@ jobs:
             cmake -S . -B build
             -DCMAKE_BUILD_TYPE=Release
             -DCMAKE_POLICY_VERSION_MINIMUM=3.5
-            "-DOPENMOQ_PICOQUIC_SOURCE_DIR=${{ github.workspace }}/third_party/picoquic"
-            "-DOPENMOQ_PICOTLS_SOURCE_DIR=${{ github.workspace }}/third_party/picotls"
             -DOPENMOQ_RUN_PICOQUIC_SMOKE_TESTS=OFF
           )
           if [ -n "${OPENSSL_ROOT_DIR:-}" ]; then
diff --git a/CMakeLists.txt b/CMakeLists.txt
diff --git a/README.es.md b/README.es.md
@@ -186,11 +186,12 @@ Consulte el [inicio rápido de la CLI](docs/quickstart.md), las [recetas de FFmp
 
 El backend opcional [moq5](https://github.com/openmoq/moq5) enruta la publicación por lotes, stdin en vivo, SRT en vivo y objetos en vivo mediante la capa de servicio de moq5. La capa de servicio gestiona la publicación del catálogo, la validación CMSF/CMAF, el gating por demanda de suscriptores, el backpressure acotado y el drenaje ordenado del transporte.
 
-CMake detecta automáticamente un checkout hermano `../moq5`. Defina `OPENMOQ_LIBMOQ_SOURCE_DIR` cuando el checkout se encuentre en otra ubicación:
+CMake obtiene la versión actual de `openmoq/moq5` `main` cuando se habilita este
+backend. Defina `OPENMOQ_LIBMOQ_SOURCE_DIR` solo para usar una fuente local u
+offline:
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.fr.md b/README.fr.md
@@ -186,11 +186,12 @@ Consultez le [démarrage rapide de la CLI](docs/quickstart.md), les [recettes FF
 
 Le backend optionnel [moq5](https://github.com/openmoq/moq5) achemine la publication batch, stdin en direct, SRT en direct et d'objets en direct via la couche de service de moq5. Cette couche gère la publication du catalogue, la validation CMSF/CMAF, le gating selon la demande des abonnés, la backpressure bornée et le drainage propre du transport.
 
-CMake détecte automatiquement un checkout frère `../moq5`. Définissez `OPENMOQ_LIBMOQ_SOURCE_DIR` lorsque le checkout se trouve ailleurs :
+CMake récupère la version actuelle de `openmoq/moq5` `main` lorsque ce backend
+est activé. Définissez `OPENMOQ_LIBMOQ_SOURCE_DIR` uniquement pour utiliser une
+source locale ou hors ligne :
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.it.md b/README.it.md
@@ -186,11 +186,12 @@ Consultare l'[avvio rapido della CLI](docs/quickstart.md), le [ricette FFmpeg](d
 
 Il backend opzionale [moq5](https://github.com/openmoq/moq5) instrada la pubblicazione batch, stdin live, SRT live e di oggetti live tramite il service tier di moq5. Il service tier gestisce la pubblicazione del catalogo, la validazione CMSF/CMAF, il gating basato sulla domanda dei subscriber, la backpressure limitata e il drain ordinato del trasporto.
 
-CMake rileva automaticamente un checkout sibling `../moq5`. Impostare `OPENMOQ_LIBMOQ_SOURCE_DIR` quando il checkout si trova altrove:
+CMake recupera la versione corrente di `openmoq/moq5` `main` quando questo
+backend è abilitato. Impostare `OPENMOQ_LIBMOQ_SOURCE_DIR` solo per usare una
+sorgente locale o offline:
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.ja.md b/README.ja.md
@@ -186,11 +186,10 @@ ffmpeg -re \
 
 任意の [moq5](https://github.com/openmoq/moq5) backend は、batch、live stdin、live SRT、live object の公開を moq5 service tier 経由で処理します。service tier は catalog publication、CMSF/CMAF validation、subscriber demand gating、bounded backpressure、graceful transport drain を担当します。
 
-CMake は sibling checkout `../moq5` を自動検出します。checkout が別の場所にある場合は `OPENMOQ_LIBMOQ_SOURCE_DIR` を設定してください。
+この backend を有効にすると、CMake は現在の `openmoq/moq5` `main` を取得します。local または offline の source override を使う場合にのみ `OPENMOQ_LIBMOQ_SOURCE_DIR` を設定してください。
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.md b/README.md
@@ -186,11 +186,11 @@ See the [CLI quick start](docs/quickstart.md), [FFmpeg recipes](docs/ffmpeg.md),
 
 The opt-in [moq5](https://github.com/openmoq/moq5) backend routes batch, live stdin, live SRT, and live-object publishing through moq5's service tier. The service tier handles catalog publication, CMSF/CMAF validation, subscriber-demand gating, bounded backpressure, and graceful transport draining.
 
-CMake auto-detects a sibling `../moq5` checkout. Set `OPENMOQ_LIBMOQ_SOURCE_DIR` when the checkout is elsewhere:
+CMake fetches current `openmoq/moq5` `main` when this backend is enabled. Set
+`OPENMOQ_LIBMOQ_SOURCE_DIR` only to use a local or offline source override:
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.pt.md b/README.pt.md
@@ -186,11 +186,12 @@ Consulte o [início rápido da CLI](docs/quickstart.md), as [receitas de FFmpeg]
 
 O backend opcional [moq5](https://github.com/openmoq/moq5) encaminha a publicação em lote, stdin ao vivo, SRT ao vivo e objetos ao vivo pela camada de serviço do moq5. Essa camada gerencia publicação de catálogo, validação CMSF/CMAF, gating pela demanda de assinantes, backpressure limitado e drenagem ordenada do transporte.
 
-O CMake detecta automaticamente um checkout irmão `../moq5`. Defina `OPENMOQ_LIBMOQ_SOURCE_DIR` quando o checkout estiver em outro local:
+O CMake obtém a versão atual de `openmoq/moq5` `main` quando este backend é
+habilitado. Defina `OPENMOQ_LIBMOQ_SOURCE_DIR` apenas para usar uma fonte local
+ou offline:
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/README.zh.md b/README.zh.md
@@ -186,11 +186,11 @@ ffmpeg -re \
 
 可选的 [moq5](https://github.com/openmoq/moq5) backend 通过 moq5 service tier 处理批量、直播 stdin、直播 SRT 和直播 object 发布。service tier 负责 catalog 发布、CMSF/CMAF 验证、基于 subscriber demand 的 gating、有限 backpressure 和有序 transport drain。
 
-CMake 会自动检测 sibling checkout `../moq5`。如果 checkout 位于其他位置，请设置 `OPENMOQ_LIBMOQ_SOURCE_DIR`：
+启用此 backend 后，CMake 会获取当前的 `openmoq/moq5` `main`。仅在使用本地或
+offline source override 时设置 `OPENMOQ_LIBMOQ_SOURCE_DIR`：
 
 ```bash
 cmake -S . -B build-libmoq \
-  -DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5 \
   -DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON
 cmake --build build-libmoq
 ctest --test-dir build-libmoq --output-on-failure
diff --git a/cmake/OpenMoqDependencies.cmake b/cmake/OpenMoqDependencies.cmake
@@ -0,0 +1,106 @@
+include_guard(GLOBAL)
+
+function(openmoq_dependency_refresh_due)
+    set(options)
+    set(one_value_args STAMP_FILE INTERVAL_HOURS NOW_EPOCH RESULT)
+    cmake_parse_arguments(ARG "${options}" "${one_value_args}" "" ${ARGN})
+
+    foreach(required_arg IN ITEMS STAMP_FILE INTERVAL_HOURS RESULT)
+        if(NOT DEFINED ARG_${required_arg} OR "${ARG_${required_arg}}" STREQUAL "")
+            message(FATAL_ERROR
+                "openmoq_dependency_refresh_due requires ${required_arg}")
+        endif()
+    endforeach()
+
+    if(NOT ARG_INTERVAL_HOURS MATCHES "^[0-9]+$")
+        message(FATAL_ERROR "INTERVAL_HOURS must be a non-negative integer")
+    endif()
+
+    if(DEFINED ARG_NOW_EPOCH AND NOT "${ARG_NOW_EPOCH}" STREQUAL "")
+        set(now_epoch "${ARG_NOW_EPOCH}")
+    else()
+        string(TIMESTAMP now_epoch "%s" UTC)
+    endif()
+
+    set(refresh_due TRUE)
+    if(EXISTS "${ARG_STAMP_FILE}")
+        file(READ "${ARG_STAMP_FILE}" last_refresh_epoch)
+        string(STRIP "${last_refresh_epoch}" last_refresh_epoch)
+        if(last_refresh_epoch MATCHES "^[0-9]+$")
+            math(EXPR refresh_interval_seconds "${ARG_INTERVAL_HOURS} * 60 * 60")
+            math(EXPR refresh_age_seconds "${now_epoch} - ${last_refresh_epoch}")
+            if(refresh_age_seconds GREATER_EQUAL 0 AND
+                    refresh_age_seconds LESS refresh_interval_seconds)
+                set(refresh_due FALSE)
+            endif()
+        endif()
+    endif()
+
+    set(${ARG_RESULT} "${refresh_due}" PARENT_SCOPE)
+endfunction()
+
+function(openmoq_record_dependency_refresh)
+    set(options)
+    set(one_value_args STAMP_FILE NOW_EPOCH)
+    cmake_parse_arguments(ARG "${options}" "${one_value_args}" "" ${ARGN})
+
+    if(NOT DEFINED ARG_STAMP_FILE OR "${ARG_STAMP_FILE}" STREQUAL "")
+        message(FATAL_ERROR "openmoq_record_dependency_refresh requires STAMP_FILE")
+    endif()
+
+    if(DEFINED ARG_NOW_EPOCH AND NOT "${ARG_NOW_EPOCH}" STREQUAL "")
+        set(now_epoch "${ARG_NOW_EPOCH}")
+    else()
+        string(TIMESTAMP now_epoch "%s" UTC)
+    endif()
+
+    get_filename_component(stamp_directory "${ARG_STAMP_FILE}" DIRECTORY)
+    file(MAKE_DIRECTORY "${stamp_directory}")
+    file(WRITE "${ARG_STAMP_FILE}" "${now_epoch}\n")
+endfunction()
+
+function(openmoq_record_dependency_refresh_if_due)
+    set(options)
+    set(one_value_args REFRESH_DUE STAMP_FILE NOW_EPOCH)
+    cmake_parse_arguments(ARG "${options}" "${one_value_args}" "" ${ARGN})
+
+    if(NOT DEFINED ARG_REFRESH_DUE)
+        message(FATAL_ERROR
+            "openmoq_record_dependency_refresh_if_due requires REFRESH_DUE")
+    endif()
+    if(NOT DEFINED ARG_STAMP_FILE OR "${ARG_STAMP_FILE}" STREQUAL "")
+        message(FATAL_ERROR
+            "openmoq_record_dependency_refresh_if_due requires STAMP_FILE")
+    endif()
+
+    if(ARG_REFRESH_DUE)
+        openmoq_record_dependency_refresh(
+            STAMP_FILE "${ARG_STAMP_FILE}"
+            NOW_EPOCH "${ARG_NOW_EPOCH}")
+    endif()
+endfunction()
+
+function(openmoq_report_git_dependency dependency_name source_dir tracking_ref)
+    find_package(Git QUIET)
+    if(NOT Git_FOUND OR NOT EXISTS "${source_dir}/.git")
+        message(STATUS
+            "OpenMOQ: ${dependency_name} source=${source_dir} ref=${tracking_ref}")
+        return()
+    endif()
+
+    execute_process(
+        COMMAND "${GIT_EXECUTABLE}" rev-parse HEAD
+        WORKING_DIRECTORY "${source_dir}"
+        RESULT_VARIABLE git_result
+        OUTPUT_VARIABLE git_head
+        ERROR_QUIET
+        OUTPUT_STRIP_TRAILING_WHITESPACE)
+    if(git_result EQUAL 0)
+        message(STATUS
+            "OpenMOQ: ${dependency_name} source=${source_dir} "
+            "ref=${tracking_ref} commit=${git_head}")
+    else()
+        message(STATUS
+            "OpenMOQ: ${dependency_name} source=${source_dir} ref=${tracking_ref}")
+    endif()
+endfunction()
diff --git a/docs/build.md b/docs/build.md
@@ -39,19 +39,37 @@ cmake --build build --target openmoq-publisher-msfts-example
 Its CMake target links only `openmoq_publisher_lib`; it does not consume moq5
 or picoquic APIs directly.
 
-## Build with Local Picoquic and Picotls
+## Managed Pico Dependencies
+
+By default, CMake owns dependency checkouts under `<build>/_deps` and follows:
+
+- `private-octopus/picoquic` `master`
+- canonical `h2o/picotls` `master`, including its recorded submodules
+- `openmoq/moq5` `main` when `OPENMOQ_USE_LIBMOQ_PUBLISHER=ON`
 
-By default, CMake looks for:
+The first configure downloads the current branch heads. Later configures use a
+successful-check timestamp in that build directory and contact upstream again
+after 24 hours. If a required initial download or scheduled refresh cannot
+reach upstream, configure fails instead of silently accepting a stale branch.
+Configure output records each resolved source directory, tracking branch, and
+commit. Run the configure command before an incremental build when you want the
+daily dependency check; CI and release builds always configure first.
+
+Set `OPENMOQ_DEPENDENCY_REFRESH_INTERVAL_HOURS` to change the interval. Setting
+it to `0` checks on every configure.
+
+## Build with Local Picoquic and Picotls
 
-- sibling `../picoquic` and `../picotls` checkouts
-- fallback: `third_party/picoquic` and `third_party/picotls`
-- fallback: `thirdparty/picoquic` and `thirdparty/picotls`
+Explicit source overrides remain available for offline development or testing a
+local dependency change. CMake never fetches, checks out, or otherwise modifies
+an override directory.
 
-If you prefer custom paths, clone picoquic and picotls to any convenient location and initialize the picotls submodules:
+Clone picoquic and canonical picotls to any convenient location and initialize
+the picotls submodules:
 
 ```bash
 git clone https://github.com/private-octopus/picoquic.git /path/to/picoquic
-git clone --recurse-submodules https://github.com/private-octopus/picotls.git /path/to/picotls
+git clone --recurse-submodules https://github.com/h2o/picotls.git /path/to/picotls
 ```
 
 Then point CMake at them:
@@ -74,8 +92,6 @@ picotls requires both `pkg-config` and OpenSSL headers and libraries. On Windows
 choco install pkgconfiglite openssl
 
 cmake -S . -B build `
-  -DOPENMOQ_PICOQUIC_SOURCE_DIR=C:\path\to\picoquic `
-  -DOPENMOQ_PICOTLS_SOURCE_DIR=C:\path\to\picotls `
   -DOPENSSL_ROOT_DIR="C:\Program Files\OpenSSL-Win64" `
   -DOPENMOQ_RUN_PICOQUIC_SMOKE_TESTS=OFF
 cmake --build build --config Release
@@ -87,9 +103,10 @@ GitHub Actions workflows set `OPENSSL_ROOT_DIR` automatically from the runner's
 ## Useful CMake Options
 
 - `-DOPENMOQ_ENABLE_PICOQUIC=ON|OFF`
-- `-DOPENMOQ_PICOQUIC_SOURCE_DIR=/path/to/picoquic`
-- `-DOPENMOQ_PICOTLS_SOURCE_DIR=/path/to/picotls`
-- `-DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5`
+- `-DOPENMOQ_DEPENDENCY_REFRESH_INTERVAL_HOURS=24`
+- `-DOPENMOQ_PICOQUIC_SOURCE_DIR=/path/to/picoquic` (explicit local override)
+- `-DOPENMOQ_PICOTLS_SOURCE_DIR=/path/to/picotls` (explicit local override)
+- `-DOPENMOQ_LIBMOQ_SOURCE_DIR=/path/to/moq5` (explicit local override)
 - `-DOPENMOQ_OPENSSL_ROOT_DIR=/path/to/openssl`
 - `-DOPENSSL_ROOT_DIR=/path/to/openssl`
 - `-DOPENMOQ_RUN_PICOQUIC_SMOKE_TESTS=ON|OFF`
@@ -98,16 +115,14 @@ GitHub Actions workflows set `OPENSSL_ROOT_DIR` automatically from the runner's
 ### Publish backend (temporary migration gate)
 
 moqxr is migrating its publish path from the legacy MoqtSession/moxygen-style
-transport onto the sibling **libmoq** service tier. While the migration is being
+transport onto the **libmoq** service tier. While the migration is being
 reviewed, the backend is selectable:
 
-- **libmoq available** — the libmoq integration code (translation, drivers,
-  tests) builds and is validated whenever CMake finds a sibling `../moq5`
-  checkout. It also accepts `../libmoq`, `third_party/moq5`,
-  `thirdparty/moq5`, or an explicit `OPENMOQ_LIBMOQ_SOURCE_DIR`; availability
-  is independent of which backend is *selected*.
 - **`-DOPENMOQ_USE_LIBMOQ_PUBLISHER=ON`** — the production `Publisher` routes
-  batch, live stdin, live SRT, and `LiveObjectSource` publishing through libmoq.
+  batch, live stdin, live SRT, and `LiveObjectSource` publishing through a
+  managed checkout of `openmoq/moq5` `main`.
+- **local libmoq override** — setting `OPENMOQ_LIBMOQ_SOURCE_DIR` builds and
+  validates that source tree even when the legacy backend remains selected.
 - **Caller-supplied catalog exception** — a `LiveObjectSource` using
   `LiveCatalogMode::kSourceObject` is routed through `MoqtSession` in either
   configuration. This preserves catalog formats such as MSFTS `"m2ts"` that
@@ -125,7 +140,7 @@ This gate is **temporary** — it will be removed once the libmoq publish path i
 accepted as the default. An injected `TransportFactory` always forces the legacy
 path regardless of this option.
 
-For a checkout outside the auto-detected locations:
+For a local checkout override:
 
 ```bash
 cmake -S . -B build-libmoq \
@@ -142,7 +157,8 @@ GitHub Actions publishes release archives for Linux, macOS, and Windows:
 - pushing a `v*` tag builds release artifacts and attaches them to the matching GitHub Release
 - running the `Release Builds` workflow manually uploads the same archives as workflow artifacts
 - manual runs can also publish a GitHub Release when you provide a `release_tag` such as `v0.1.0`
-- both CI and release workflows check out `private-octopus/picoquic` plus `private-octopus/picotls`, so published binaries include the picoquic transport path
+- CI and release use the managed dependency flow, so each clean job resolves
+  picoquic `master` and canonical picotls `master` during configure
 - Linux and macOS archives are `.tar.gz` and contain `openmoq-publisher`, `libopenmoq_publisher.a`, `include/`, `docs/`, `README.md`, and `LICENSE`
 - Windows archives are `.zip` and contain `openmoq-publisher.exe`, `openmoq_publisher.lib`, `include/`, `docs/`, `README.md`, and `LICENSE`
 
@@ -156,4 +172,5 @@ GitHub Actions builds and tests the project on:
   `openmoq-publisher-libmoq-translation-tests` target
 
 Every lane runs CMake configure, the full default build, a static-library
-existence check, and CTest.
+existence check, and CTest. The libmoq lane also resolves `openmoq/moq5` `main`
+during configure.
diff --git a/docs/testing.md b/docs/testing.md
diff --git a/tests/cmake/dependency_refresh_test.cmake b/tests/cmake/dependency_refresh_test.cmake
