Make fork compatible with DuckDB v1.5.3 (#21)

* Publish DuckLake release binaries on tags (#7)

* ci: bigger runner for build, bigger ccache for tests (#8)

Speed up CI via two independent levers:

1. MainDistributionPipeline.yml: route the linux_amd64 / linux_arm64
   build matrix entries to 16-core Depot managed runners
   (depot-ubuntu-24.04-16 / depot-ubuntu-24.04-arm-16) via the upstream
   workflow's runners JSON input. Cuts the build from ~25-30min to
   ~5-10min, including cold-cache cases.

   Requires PostHog/ducklake to be added to the Depot GitHub App
   installation; until then the build jobs will queue-time-out at 10m.

2. Five per-PR test workflows (Catalogs, ConfigTests, DeletionVectors,
   MinIO, NoInline): bump hendrikmuhs/ccache-action max-size from the
   500MB default to 1.5G. DuckDB's full release build produces 1-2 GB
   of object files, so the default forces aggressive LRU eviction and
   keeps effective hit rate at 30-50%. 1.5G fits most of the build,
   pushing warm-cache hit rates into the 80-90% range.

   5 caches x 1.5G = 7.5 GB, leaving headroom in the 10 GB GitHub
   Actions cache budget per repo. Cold-cache cases (DuckDB submodule
   bumps) will still take ~25-30min but are infrequent.

Debug.yml is now nightly-only after #9 and is left alone.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

* [codex] Route Postgres DuckLake metadata through passthrough (#5)

* Route Postgres metadata reads through passthrough

* Fix Postgres metadata passthrough regressions

* Keep Postgres thread cache setting test-scoped

* Fix Postgres test pool setting init order

* Update postgres scanner for pool defaults

* Route Postgres global stats through passthrough (#11)

* Remove DuckDB extension deploy job

* Pin inlined table registry to snapshot schema

* Fix duplicate Query(string&) after v1.5.3 rebase

Rebasing the Postgres metadata passthrough patch (#5) onto DuckDB
v1.5.3 replayed an old DuckLakeMetadataManager::Query(string&) body
(returning transaction.Query(query)) as a second definition alongside
upstream's new implementation, which uses SubstituteCatalogPlaceholders
+ transaction.ExecuteRaw. Drop the stale duplicate; upstream's version
is correct for the v1.5.3 metadata-manager architecture.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: pin GitHub Actions to commit SHAs

The v1.5.3 workflows inherited from upstream use tag/branch refs
(actions/checkout@v4, ccache-action@main, etc.), which the PostHog org
ruleset rejects ("all actions must be pinned to a full-length commit
SHA"). Pin every action to its commit SHA, reusing the SHAs vetted in
#17 on main where applicable. No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix infinite recursion in Postgres metadata passthrough

PostgresMetadataManager::ExecuteQuery builds a `CALL postgres_query(...)`
passthrough statement and ran it via transaction.Query(). In DuckDB
v1.5.3's metadata architecture transaction.Query() delegates back to
metadata_manager->Query(), i.e. PostgresMetadataManager::Query(), which
calls ExecuteQuery() again and wraps the statement in yet another
CALL postgres_query(...) — recursing without bound and hanging the
"Test Postgres" suite (SQLite, which has no passthrough, was unaffected).

Run the fully-formed passthrough call via transaction.ExecuteRaw() so it
goes straight to the metadata connection without re-entering the manager.
This is the load-bearing change that #19 made (as RawQuery) and that was
lost when #19 was skipped during the v1.5.3 rebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Re-derive metadata query-intent split (#19) on v1.5.3 architecture

The Postgres metadata passthrough (#5/#11) depends on a read/write/raw
routing split that #19 introduced and that was dropped during the v1.5.3
rebase. v1.5.3 had collapsed all metadata access onto transaction.Query,
which for a Postgres backend wrongly routed everything through the
postgres_query passthrough — breaking DDL (uuid()), the metadata-exists
probe, local utility queries (ATTACH, duckdb_secrets), and recursing on
the passthrough CALL itself.

Restore the split, adapted to v1.5.3's manager-side substitution and
ExecuteRaw executor:

- Transaction-level Execute/SnapshotQuery/CurrentQuery/RawQuery run RAW on
  the metadata connection (DuckDB executes against the attached Postgres
  catalog, handling uuid()/LIST/STRUCT and queries that embed
  postgres_query CTEs).
- Metadata-manager-level Execute/SnapshotQuery/CurrentQuery do the
  postgres_execute/postgres_query passthrough, used (unqualified) only by
  the snapshot-pinned file/stats reads that benefit from server-side
  execution (#5/#11). Simple committed reads (e.g. global stats, inlined
  file deletions) also use the manager passthrough.
- Call sites relabeled by intent: writes -> Execute, snapshot reads ->
  SnapshotQuery, current-state reads -> CurrentQuery, local utilities ->
  RawQuery. Notably ReadInlinedFileDeletions and other inlined-deletion
  reads were running through Execute (postgres_execute returns no rows),
  which double-applied inlined deletions.
- Port the check_metadata_query_intent.py guard and wire it into CI.

Verified locally against DuckDB, SQLite and Postgres backends
(postgres_query/postgres_execute via Docker): all three suites green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Materialize embedded postgres_query reads (v1.5.3 streaming limit)

DuckDB v1.5.3 rejects multiple streaming scans, or a streaming scan
alongside an insert/CTAS, in one query. The Postgres metadata reads that
embed postgres_query() run raw, so an un-materialized postgres_query
streams:

- GetLatestSnapshotQuery() is evaluated during write statements, so its
  streaming postgres_query scan collided with the insert ("Failed to
  query most recent snapshot ... Multiple streaming scans"). Wrap it in a
  MATERIALIZED CTE.
- The file-column-stats CTE used NOT MATERIALIZED for single-reference
  columns; multiple such columns produced multiple streaming
  postgres_query scans. Always mark it MATERIALIZED.

Negligible cost (small metadata results). Only surfaces with a streaming
postgres_scanner build (CI); a materializing local build masked it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Route GetSnapshot through CALL passthrough, not streaming postgres_query

The MATERIALIZED-CTE attempt didn't help: an embedded
"SELECT * FROM postgres_query(...)" is a streaming scan, and GetSnapshot()
runs during write statements, so it tripped DuckDB v1.5.3's "streaming
scan + insert in the same query" restriction. Wrapping it in a
MATERIALIZED CTE just turned it into "streaming scan + CTAS".

Use the metadata-manager passthrough instead, which executes
CALL postgres_query(...) as a separate statement returning a materialized
result (the same path GetSnapshot(at_clause) already uses and that global
stats use successfully in CI):

- GetSnapshot(): call the manager-level CurrentQuery() (passthrough)
  instead of transaction.CurrentQuery() (raw embedded scan).
- Drop the Postgres GetLatestSnapshotQuery() override so the plain base
  SELECT is used; the CALL passthrough wraps it.
- Revert the file-column-stats CTE back to its original MATERIALIZED hint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: build via PostHog/extension-ci-tools fork (pinned actions)

The PostHog org ruleset rejects any unpinned action ref, enforced
transitively into reusable workflows. duckdb/extension-ci-tools uses
unpinned internal actions (actions/checkout@v4, ...), so every run
startup-failed at "Prepare all required actions".

Point the distribution call at the PostHog extension-ci-tools fork,
whose main (d3e0ee09) pins every internal action to a SHA and exposes
the `runners` input, and pull ci-tools makefiles from the same fork/ref
via override_ci_tools_repository.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: build linux_arm64 on GitHub-hosted runner

The depot arm64 runner was unresponsive (job lost at the 10-minute
runner timeout). This repo is public, so use the free GitHub-hosted
ubuntu-24.04-arm runner for arm64; keep amd64 on the depot runner.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Pin metadata postgres_scanner to single-threaded (materialize scans)

DuckLake metadata reads (e.g. GetCatalogForSnapshot) scan several attached
Postgres tables in one statement using DuckDB LIST/STRUCT syntax, so they
must run on the metadata connection rather than server-side. DuckDB v1.5.3
rejects multiple streaming postgres_scanner scans (and streaming + insert)
in a single query, which these multi-table reads triggered.

postgres_scanner only streams a scan when max_threads > 1, where
max_threads = pages / pg_pages_per_task. After attaching the Postgres
metadata catalog, set pg_pages_per_task to a very large value so
max_threads is pinned to 1 and these reads materialize instead of
streaming. Metadata results are tiny, so there is no throughput cost.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Align postgres_scanner to upstream v1.5.3 pin (c0e9256)

#5 bumped postgres_scanner forward to dd71d196 (the "Connection pool"
rework, which adds the database-connector submodule) on the DuckDB 1.5.2
line. That newer scanner is incompatible with DuckDB v1.5.3's restriction
on streaming postgres scans: DuckLake's multi-table metadata reads
(GetCatalogForSnapshot, ...) failed with "Multiple streaming scans or
streaming scans + CTAS / insert in the same query are not currently
supported".

Pin postgres_scanner to c0e9256 -- the exact version upstream DuckLake's
v1.5-variegata (DuckDB v1.5.3) is tested against -- and drop the
ineffective pg_pages_per_task workaround. Revisit the connection-pool
scanner bump separately against a v1.5.3-compatible scanner if needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Pin postgres_scanner to the DuckDB v1.5.3 release version (6b2b12c)

Root cause of the remaining Postgres CI failures: CI built an older
postgres_scanner that streams multi-table scans, which DuckDB v1.5.3
rejects ("Multiple streaming scans or streaming scans + CTAS / insert in
the same query are not currently supported") for DuckLake's metadata
reads (GetCatalogForSnapshot scans 4 metadata tables in one query).

6b2b12c is the postgres_scanner the DuckDB v1.5.3 extension release ships;
it materializes those scans instead of streaming. It is 29 commits ahead
of #5's dd71d196 (so it keeps the connection-pool work) and 79 ahead of
c0e9256. Verified: the full Postgres test suite passes locally against
this scanner (428 test cases, 0 failures), alongside the green DuckDB and
SQLite suites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: test Postgres against the prebuilt scanner, not a source build

Root cause of the persistent Postgres CI failures: the test job set
ENABLE_POSTGRES_SCANNER=ON and statically built postgres_scanner from
source. That source build streams multi-table postgres scans, which
DuckDB v1.5.3 rejects ("Multiple streaming scans or streaming scans +
CTAS / insert in the same query are not currently supported") for
DuckLake's metadata reads (GetCatalogForSnapshot scans 4 tables at once).
The officially released postgres_scanner materializes those scans.

Stop building postgres_scanner from source for the test job; install the
released extension (INSTALL postgres FROM core) instead, exactly as a real
deployment would load it. The full Postgres suite passes locally with the
prebuilt extension (428 test cases, 0 failures). SQLite/Quack still build
from source as before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(quack): route catalog-load reads through the quack passthrough

GetCatalogForSnapshot runs six snapshot-pinned reads via the raw
transaction-level SnapshotQuery, which scans the attached metadata
catalog directly. For the quack backend that produces multiple
streaming quack_query scans in one statement, and quack's optimizer
rejects that ("Multiple streaming scans ... not currently supported"),
failing 350/394 tests in the Test Quack step.

Add a SnapshotCatalogQuery seam on the metadata manager. The base
implementation keeps the existing raw behavior (correct for the DuckDB
backend, and for Postgres where postgres_scanner materializes the
multi-table scan). QuackMetadataManager overrides it to route through
CALL quack_query_by_name, which executes the DuckDB SQL server-side in
a real DuckDB and returns a materialized result.

Verified locally on the DuckDB, Postgres, and SQLite backends; quack is
verified via CI (cannot build the quack variant locally). Intent guard
still passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(quack): route all multi-table metadata reads through the passthrough

The first CI run after the GetCatalogForSnapshot fix moved the failure
forward: passes went 44 -> 113, the table-information error disappeared,
and the next raw multi-table reads surfaced the same quack streaming-scan
error (500 data-file-list, 220 extended-file-list, 16 column lookup, 16
orphan-cleanup).

Generalize the seam introduced for GetCatalogForSnapshot: add a
CurrentCatalogQuery counterpart to SnapshotCatalogQuery (both default to
the raw transaction read; quack overrides both to its quack_query_by_name
passthrough) and route the remaining raw multi-table reads through them:
GetFilesForTable, the extended file list, IsColumnCreatedWithTable,
GetOrphanFilesForCleanup, and GetTableSizes.

Single-table raw reads are left as-is: a single streaming quack scan is
allowed by quack's optimizer. The default keeps DuckDB/Postgres/SQLite on
the raw path (verified: stats, compaction, alter, remove_orphans,
add_files pass on all three backends). Intent guard still passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(quack): route metadata cleanup through passthrough

* ci: clarify postgres_scanner.cmake is the local/source-build path only

CI no longer builds postgres_scanner from source (the Postgres test job
installs the released extension via `INSTALL postgres FROM core`), so the
GIT_TAG pin here is only exercised by local/manual builds that set
ENABLE_POSTGRES_SCANNER. Document that and why the pin matters, so the
file doesn't read as the active CI scanner version.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: skip flaky Quack concurrent catalog creation

* test: skip Quack deletion inlining retry flake

---------

Co-authored-by: Bill Guowei Yang <bill@posthog.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Author: 3 people

.github/config/extensions/postgres_scanner.cmake

@@ -1,8 +1,15 @@
-# postgres_scanner needs DONT_LINK because it depends on libpq/OpenSSL
+# Source-build of postgres_scanner, used only for LOCAL/manual builds that set ENABLE_POSTGRES_SCANNER
+# (see extension_config.cmake). CI does NOT build it from source -- the Postgres test job installs the
+# released extension via `INSTALL postgres FROM core` (see .github/workflows/Catalogs.yml), because a
+# source build streams multi-table scans, which DuckDB v1.5.3 rejects.
+#
+# When building from source, pin to the postgres_scanner shipped with the DuckDB v1.5.3 release
+# (6b2b12c), which materializes those scans. postgres_scanner needs DONT_LINK (depends on libpq/OpenSSL).
 if (NOT MINGW AND NOT ${WASM_ENABLED})
     duckdb_extension_load(postgres_scanner
             DONT_LINK
             GIT_URL https://github.com/duckdb/duckdb-postgres
-            GIT_TAG c0e9256a60371a062e8d6cfc4045c71a317b5ee8
+            GIT_TAG 6b2b12cad3afef61e8a4637e714e8a88895fed1a
+            SUBMODULES database-connector
             )
 endif()

.github/workflows/Catalogs.yml

@@ -24,7 +24,10 @@ jobs:
       GEN: Ninja
       VCPKG_TOOLCHAIN_PATH: ${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake
       ENABLE_SQLITE_SCANNER: ON
-      ENABLE_POSTGRES_SCANNER: ON
+      # Do NOT statically build postgres_scanner from source: the source build streams multi-table
+      # postgres scans, which DuckDB v1.5.3 rejects ("Multiple streaming scans ..."). The official
+      # prebuilt postgres_scanner (installed in the Test Postgres step) materializes them. The full
+      # Postgres suite passes locally with the prebuilt extension.
       ENABLE_QUACK: ON
       BUILD_EXTENSION_TEST_DEPS: full
 
@@ -35,7 +38,7 @@ jobs:
         sudo apt-get install -y -qq build-essential cmake ninja-build ccache python3
 
     - name: Setup PostgreSQL for Linux/macOS/Windows
-      uses: ikalnytskyi/action-setup-postgres@v7
+      uses: ikalnytskyi/action-setup-postgres@10ab8a56cc77b4823c2bfa57b1d4dd5605ef0481 # v7
       with:
         postgres-version: '15'
         username: 'postgres'
@@ -44,7 +47,7 @@ jobs:
         port: '5432'
         ssl: true
 
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
       with:
         fetch-depth: 0
         submodules: 'true'
@@ -55,13 +58,14 @@ jobs:
         git checkout $(cat ../.github/duckdb-version)
 
     - name: Setup Ccache
-      uses: hendrikmuhs/ccache-action@main
+      uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
       with:
         key: ${{ github.job }}
         save: ${{ github.ref == 'refs/heads/main' || github.repository != 'duckdb/ducklake' }}
+        max-size: "1.5G"
 
     - name: Setup vcpkg
-      uses: lukka/run-vcpkg@v11.1
+      uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
       with:
         vcpkgGitCommitId: 84bab45d415d22042bd0b9081aea57f362da3f35
 
@@ -88,15 +92,11 @@ jobs:
       env:
         PGSERVICE: postgres
       run: |
-        # Pre-install postgres_scanner so LOAD finds it without autoloading
-        # (autoloading is disabled to prevent loading wrong ducklake version)
-        # Find the actual extension folder - CMake uses the git hash for dev builds
-        # and the version string for release builds
-        REPO_DIR=$(dirname $(find build/release/repository -name "postgres_scanner.duckdb_extension" | head -1))
-        DUCKDB_VER=$(basename $(dirname "$REPO_DIR"))
-        EXT_DIR="$HOME/.duckdb/extensions/$DUCKDB_VER/${{ matrix.arch }}"
-        mkdir -p "$EXT_DIR"
-        cp "$REPO_DIR/postgres_scanner.duckdb_extension" "$EXT_DIR/"
+        # Install the official prebuilt postgres_scanner for this DuckDB version (autoloading is
+        # disabled in the test config). We deliberately do NOT build it from source: the source
+        # build streams multi-table postgres scans, which DuckDB v1.5.3 rejects, whereas the
+        # released extension materializes them.
+        build/release/duckdb -unsigned -c "INSTALL postgres FROM core; LOAD postgres;"
         createdb ducklakedb
         build/release/test/unittest --test-config test/configs/postgres.json --test-dir ./ "test/sql/*"
 

.github/workflows/ConfigTests.yml

@@ -30,24 +30,29 @@ jobs:
         sudo apt-get update -y -qq
         sudo apt-get install -y -qq build-essential cmake ninja-build ccache python3
 
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
       with:
         fetch-depth: 0
         submodules: 'true'
 
+    - name: Check metadata query intent
+      run: |
+        python3 scripts/check_metadata_query_intent.py
+
     - name: Checkout DuckDB to version
       run: |
         cd duckdb
         git checkout $(cat ../.github/duckdb-version)
 
     - name: Setup Ccache
-      uses: hendrikmuhs/ccache-action@main
+      uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
       with:
         key: ${{ github.job }}
         save: ${{ github.ref == 'refs/heads/main' || github.repository != 'duckdb/ducklake' }}
+        max-size: "1.5G"
 
     - name: Setup vcpkg
-      uses: lukka/run-vcpkg@v11.1
+      uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
       with:
         vcpkgGitCommitId: a42af01b72c28a8e1d7b48107b33e4f286a55ef6
 

.github/workflows/Debug.yml

@@ -40,7 +40,7 @@ jobs:
         sudo apt-get update -y -qq
         sudo apt-get install -y -qq build-essential cmake ninja-build ccache python3
 
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
       with:
         fetch-depth: 0
         submodules: 'true'
@@ -51,13 +51,13 @@ jobs:
         git checkout $(cat ../.github/duckdb-version)
 
     - name: Setup Ccache
-      uses: hendrikmuhs/ccache-action@main
+      uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
       with:
         key: ${{ github.job }}
         save: ${{ github.ref == 'refs/heads/main' || github.repository != 'duckdb/ducklake' }}
 
     - name: Setup vcpkg
-      uses: lukka/run-vcpkg@v11.1
+      uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
       with:
         vcpkgGitCommitId: a42af01b72c28a8e1d7b48107b33e4f286a55ef6
 

.github/workflows/DeletionVectors.yml

@@ -30,7 +30,7 @@ jobs:
         sudo apt-get update -y -qq
         sudo apt-get install -y -qq build-essential cmake ninja-build ccache python3
 
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
       with:
         fetch-depth: 0
         submodules: 'true'
@@ -41,13 +41,14 @@ jobs:
         git checkout $(cat ../.github/duckdb-version)
 
     - name: Setup Ccache
-      uses: hendrikmuhs/ccache-action@main
+      uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
       with:
         key: ${{ github.job }}
         save: ${{ github.ref == 'refs/heads/main' || github.repository != 'duckdb/ducklake' }}
+        max-size: "1.5G"
 
     - name: Setup vcpkg
-      uses: lukka/run-vcpkg@v11.1
+      uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
       with:
         vcpkgGitCommitId: a42af01b72c28a8e1d7b48107b33e4f286a55ef6
 

.github/workflows/MainDistributionPipeline.yml

@@ -18,7 +18,7 @@ jobs:
     outputs:
       duckdb_version: ${{ steps.version.outputs.duckdb_version }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           sparse-checkout: .github
       - id: version
@@ -27,20 +27,67 @@ jobs:
   duckdb-next-build:
     name: Build extension binaries
     needs: get-duckdb-version
-    uses: duckdb/extension-ci-tools/.github/workflows/_extension_distribution.yml@main
+    # PostHog fork of extension-ci-tools: internal actions are pinned to SHAs.
+    # The org ruleset rejects unpinned action refs transitively (even inside a
+    # SHA-referenced reusable workflow), so duckdb/extension-ci-tools cannot run
+    # here. The fork's main also exposes the `runners` input.
+    uses: PostHog/extension-ci-tools/.github/workflows/_extension_distribution.yml@d3e0ee09e2e392941f2e390e6cbab108a095daa4
     with:
       duckdb_version: ${{ needs.get-duckdb-version.outputs.duckdb_version }}
-      ci_tools_version: main
+      ci_tools_version: d3e0ee09e2e392941f2e390e6cbab108a095daa4
+      override_ci_tools_repository: PostHog/extension-ci-tools
       extension_name: ducklake
+      # arm64 on GitHub-hosted (public repo, free); the depot arm64 runner
+      # was unresponsive (job lost at the 10-min runner timeout). amd64 stays
+      # on the working depot runner.
+      runners: |
+        {"linux_amd64": "depot-ubuntu-24.04-16", "linux_arm64": "ubuntu-24.04-arm"}
 
-  duckdb-next-deploy:
-    name: Deploy extension binaries
+  publish-github-release:
+    name: Publish GitHub Release assets
+    runs-on: ubuntu-latest
     needs: [get-duckdb-version, duckdb-next-build]
-    uses: duckdb/extension-ci-tools/.github/workflows/_extension_deploy.yml@main
-    secrets: inherit
-    with:
-      extension_name: ducklake
-      duckdb_version: ${{ needs.get-duckdb-version.outputs.duckdb_version }}
-      ci_tools_version: main
-      deploy_latest: ${{ startsWith(github.ref, 'refs/heads/v') || github.ref == 'refs/heads/main' }}
-      deploy_versioned: ${{ startsWith(github.ref, 'refs/heads/v') || github.ref == 'refs/heads/main' }}
+    if: startsWith(github.ref, 'refs/tags/')
+    permissions:
+      contents: write
+      actions: read
+    env:
+      GH_TOKEN: ${{ github.token }}
+      GH_REPO: ${{ github.repository }}
+      RELEASE_TAG: ${{ github.ref_name }}
+      DUCKDB_VERSION: ${{ needs.get-duckdb-version.outputs.duckdb_version }}
+    steps:
+      - name: Download linux amd64 extension
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          name: ducklake-${{ env.DUCKDB_VERSION }}-extension-linux_amd64
+          path: release-assets/linux_amd64
+
+      - name: Download linux arm64 extension
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          name: ducklake-${{ env.DUCKDB_VERSION }}-extension-linux_arm64
+          path: release-assets/linux_arm64
+
+      - name: Stage release assets
+        run: |
+          mkdir -p release-assets/staged
+          cp release-assets/linux_amd64/ducklake.duckdb_extension release-assets/staged/ducklake-linux-amd64.duckdb_extension
+          cp release-assets/linux_arm64/ducklake.duckdb_extension release-assets/staged/ducklake-linux-arm64.duckdb_extension
+          cd release-assets/staged
+          sha256sum ducklake-linux-amd64.duckdb_extension ducklake-linux-arm64.duckdb_extension > SHA256SUMS
+
+      - name: Create or update GitHub Release
+        run: |
+          if ! gh release view "$RELEASE_TAG" >/dev/null 2>&1; then
+            gh release create "$RELEASE_TAG" \
+              --target "$GITHUB_SHA" \
+              --title "$RELEASE_TAG" \
+              --notes "PostHog DuckLake extension binaries for DuckDB ${DUCKDB_VERSION}."
+          fi
+
+          gh release upload "$RELEASE_TAG" \
+            release-assets/staged/ducklake-linux-amd64.duckdb_extension \
+            release-assets/staged/ducklake-linux-arm64.duckdb_extension \
+            release-assets/staged/SHA256SUMS \
+            --clobber

.github/workflows/MinIO.yml

@@ -39,7 +39,7 @@ jobs:
           sudo apt-get remove -y cmake cmake-data
           sudo apt-get install --allow-downgrades -y -qq 'cmake=3.*' 'cmake-data=3.*'
 
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           fetch-depth: 0
           submodules: 'true'
@@ -55,13 +55,15 @@ jobs:
           cmake --version
 
       - name: Setup vcpkg
-        uses: lukka/run-vcpkg@v11.1
+        uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
         with:
           vcpkgGitCommitId: 84bab45d415d22042bd0b9081aea57f362da3f35
 
       - name: Setup Ccache
-        uses: hendrikmuhs/ccache-action@main
+        uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
         continue-on-error: true
+        with:
+          max-size: "1.5G"
 
       - name: Build extension
         env:

.github/workflows/NoInline.yml

@@ -30,7 +30,7 @@ jobs:
         sudo apt-get update -y -qq
         sudo apt-get install -y -qq build-essential cmake ninja-build ccache python3
 
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
       with:
         fetch-depth: 0
         submodules: 'true'
@@ -41,13 +41,14 @@ jobs:
         git checkout $(cat ../.github/duckdb-version)
 
     - name: Setup Ccache
-      uses: hendrikmuhs/ccache-action@main
+      uses: hendrikmuhs/ccache-action@d62db5f07c26379fc4b4e0916f098a92573c3b03 # v1.2.23
       with:
         key: ${{ github.job }}
         save: ${{ github.ref == 'refs/heads/main' || github.repository != 'duckdb/ducklake' }}
+        max-size: "1.5G"
 
     - name: Setup vcpkg
-      uses: lukka/run-vcpkg@v11.1
+      uses: lukka/run-vcpkg@abed23940f9d7bc267b0e1a21ee7b699a3794baa # v11.1
       with:
         vcpkgGitCommitId: a42af01b72c28a8e1d7b48107b33e4f286a55ef6