Skip to content

Commit 755cfe9

Browse files
authored
QVAC-17422: Normalize addon prebuild workflows around composite actions (#1760)
* infra: switch workflow to use composite actions for setup * Apply normalization to other prebuilds * Standardize nmtcpp android setup * infra: drop dead nmtcpp prebuild patching steps These steps all patched `third-party/bergamot-translator/...` paths, which no longer exist since bergamot-translator moved to the qvac-registry-vcpkg registry as a prebuilt vcpkg port. Every step was guarded by `if [ -f ... ]` and silently no-opped on every run. Removed: - clang-cl / C++20 / CMake-version / PCRE2 / std::result_of / -march=native source patches against marian-dev, sentencepiece, clog, cpuinfo, ssplit and intgemm - `Build PCRE2 first for Windows` post-generate workaround - `submodules: recursive` on checkout (package has no .gitmodules) Kept `Create config.cmake for Bergamot backend` (consumed by the addon's CMakeLists.txt) and `Replace bare-make's CMake` on ubuntu-22.04. Workflow: 514 → 247 lines. Any port-level patches still needed for marian-dev & friends should live in the bergamot-translator vcpkg portfile in qvac-registry-vcpkg, not in CI. * infra: drop dead nmtcpp CI scripts and bare-make cmake workaround - Remove packages/qvac-lib-infer-nmtcpp/ci/*. These scripts are not referenced from any workflow, script, or doc. Running contributor-editable shell scripts during prebuilds would be a supply-chain risk (the workflow has PAT_TOKEN and AWS OIDC role access). Two of them also patched an MLC/TVM vendor path that no longer exists; the third was a standalone manual integration-test helper tied to retired `qvac-ext-lib-mlc` runs. - Remove the `Replace bare-make's CMake with compatible version` step (ubuntu-22.04). It existed to downgrade bare-make's bundled CMake 4.x so that marian-dev's cmake_minimum_required(VERSION 3.1) would still parse. Since bergamot-translator is now a prebuilt vcpkg port from qvac-registry-vcpkg, those CMake files are never processed by bare-make; the only CMakeLists run here is the addon's own (requires 3.25). Workflow: 247 → 222 lines. Other prebuild workflows were audited for similar patches against vcpkg-managed deps and are already clean. * infra: re-pin composite-actions SHA and finalize nmtcpp normalization Two related changes for the prebuild workflow normalization branch. Re-pin composite actions to the squash-merged SHA. PR #1742 landed on main as a single squash-merge, so the pre-squash SHA the prebuilds were pinned against (3d48906) is no longer reachable from main. Update all 9 prebuilds-*.yml to 1d9b216 a6d8 across setup-vcpkg, setup-build-host, setup-apple-clang, and strip-prebuilds. Finalize nmtcpp prebuild normalization: - Drop "Create config.cmake for Bergamot backend". All three cache vars it set are dead post bergamot-translator-vcpkg migration: USE_BERGAMOT is already the addon CMakeLists.txt default; nothing in the addon adds -DUSE_INTGEMM so the bergamot.cpp #ifdefs never compile in; BUILD_ARCH was historically marian-dev's, now baked into the vcpkg port and unreferenced from the addon. - Drop "Print run state" debug step (no peer has it). - Drop "Configure git" PAT URL rewrite. The package has zero git-protocol deps; post-OSS this is the same dead config that #1591 stripped repo-wide. - Add the missing strip-prebuilds composite-action call. - Replace hardcoded `env.PKG_DIR` with a `workdir` workflow input on both triggers, matching 6 of 9 peer prebuilds. Both nmtcpp callers invoke without `workdir`, so the default keeps them working. * infra: align onnx prebuilds with convention * chore: drop unused fmt and spdlog vcpkg deps from tts Both deps have been declared in the TTS package manifest since the initial monorepo import (fede244), as transitive needs of the piper-based TTS engine. When TTS migrated from piper to chatterbox / supertonic, piper, piper-phonemize, and espeak-ng were correctly removed but fmt and spdlog were left behind. The current TTS source has zero consumers for either library: no fmt:: / spdlog:: usage in any .cpp or .hpp file, no find_package(fmt) or find_package(spdlog) in CMakeLists.txt, no fmt::fmt or spdlog::spdlog in target_link_libraries. Drop both from vcpkg.json and from the upstream-microsoft/vcpkg registry overlay in vcpkg-configuration.json. Also drop the macOS-only "Unlink Homebrew fmt" step in the TTS prebuild workflow. Introduced in #170 to defend against Homebrew fmt 12.x headers shadowing vcpkg fmt 11.x on Intel Macs, it has been emitting "No such keg" annotations on every macOS job since GitHub stopped pre-installing fmt on the macos-15 runner images. With no fmt consumer in TTS, there is nothing to protect. NOTICE regeneration left for a follow-up.
1 parent f568d00 commit 755cfe9

14 files changed

Lines changed: 189 additions & 1525 deletions

.github/workflows/prebuilds-lib-infer-diffusion.yml

Lines changed: 13 additions & 110 deletions
Original file line numberDiff line numberDiff line change
@@ -80,24 +80,6 @@ jobs:
8080
name: ${{ matrix.platform }}-${{ matrix.arch }}${{ matrix.tags }}
8181

8282
steps:
83-
- if: ${{ matrix.platform == 'android' }}
84-
name: Select NDK
85-
run: |
86-
echo "ANDROID_NDK=$ANDROID_NDK_LATEST_HOME" >> $GITHUB_ENV
87-
echo "ANDROID_NDK_HOME=$ANDROID_NDK_LATEST_HOME" >> $GITHUB_ENV
88-
echo "ANDROID_NDK_ROOT=$ANDROID_NDK_LATEST_HOME" >> $GITHUB_ENV
89-
90-
- name: print env
91-
run: printenv
92-
93-
- if: ${{ startsWith(matrix.os, 'ubuntu-') }}
94-
name: Maximize build space
95-
run: |
96-
sudo docker image prune --all --force
97-
sudo rm -rf /opt/hostedtoolcache/CodeQL
98-
sudo rm -rf /opt/ghc
99-
sudo rm -rf /usr/share/dotnet
100-
10183
- name: Echo workflow inputs
10284
shell: bash
10385
env:
@@ -116,19 +98,11 @@ jobs:
11698
echo "Matrix arch: $MATRIX_ARCH"
11799
echo "Matrix tags: $MATRIX_TAGS"
118100
119-
- if: ${{ startsWith(matrix.os, 'ubuntu-') }}
120-
name: Install llvm-19
121-
run: |
122-
wget -q https://apt.llvm.org/llvm.sh
123-
chmod +x llvm.sh
124-
sudo ./llvm.sh 19 all
125-
126-
- if: ${{ matrix.os == 'windows-2022' }}
127-
name: Configure windows runner
128-
shell: powershell
129-
run: |
130-
git config --system core.longpaths true
131-
choco upgrade llvm
101+
- name: Setup build host
102+
uses: tetherto/qvac/.github/actions/setup-build-host@1d9b2165867d03c6edd675e402ee101a5d48a6d8
103+
with:
104+
platform: ${{ matrix.platform }}
105+
arch: ${{ matrix.arch }}
132106

133107
- name: Checkout repository
134108
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # 6.0.2
@@ -142,8 +116,8 @@ jobs:
142116
with:
143117
role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN }}
144118

145-
- name: Setup vcpkg cache
146-
uses: tetherto/qvac/.github/actions/setup-vcpkg-cache@0bbdca93da303a0b1634ba14a89cec085621078d
119+
- name: Setup vcpkg
120+
uses: tetherto/qvac/.github/actions/setup-vcpkg@1d9b2165867d03c6edd675e402ee101a5d48a6d8
147121
env:
148122
MODEL_S3_BUCKET: ${{ secrets.MODEL_S3_BUCKET }}
149123
with:
@@ -153,58 +127,8 @@ jobs:
153127
- name: Setup Bare tooling
154128
uses: tetherto/qvac/.github/actions/setup-bare-tooling@0bbdca93da303a0b1634ba14a89cec085621078d
155129

156-
- if: ${{ matrix.os == 'windows-2022' && matrix.arch == 'arm64' }}
157-
name: Rename win sdk folder to disable it
158-
shell: powershell
159-
run: |
160-
Rename-Item -Path "C:\Program Files (x86)\Windows Kits\10\Include\10.0.26100.0" -NewName "C:\Program Files (x86)\Windows Kits\10\Include\old-10.0.26100.0"
161-
Rename-Item -Path "C:\Program Files (x86)\Windows Kits\10\bin\10.0.26100.0" -NewName "C:\Program Files (x86)\Windows Kits\10\bin\old-10.0.26100.0"
162-
Rename-Item -Path "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.26100.0" -NewName "C:\Program Files (x86)\Windows Kits\10\Lib\old-10.0.26100.0"
163-
164-
- if: ${{ startsWith(matrix.os, 'macos-') }}
165-
name: Install vcpkg in macos
166-
shell: bash
167-
run: |
168-
cd ..
169-
git clone --branch 2025.12.12 --single-branch https://github.com/microsoft/vcpkg.git
170-
cd vcpkg && ./bootstrap-vcpkg.sh -disableMetrics
171-
VCPKG_ROOT=$(pwd)
172-
echo "VCPKG_ROOT=$VCPKG_ROOT" >> $GITHUB_ENV
173-
echo "$VCPKG_ROOT" >> $GITHUB_PATH
174-
175-
- if: ${{ startsWith(matrix.os, 'ubuntu-') }}
176-
name: Configure vcpkg in linux
177-
run: echo "VCPKG_ROOT=$VCPKG_INSTALLATION_ROOT" >> $GITHUB_ENV
178-
179-
- if: ${{ matrix.os == 'windows-2022' }}
180-
name: Configure vcpkg in windows
181-
shell: powershell
182-
run: echo ("VCPKG_ROOT=$env:VCPKG_INSTALLATION_ROOT" -replace '\\', '/') >> $env:GITHUB_ENV
183-
184-
- if: ${{ matrix.os == 'windows-2022' }}
185-
name: Configure cmake generator in windows
186-
shell: powershell
187-
run: echo "CMAKE_GENERATOR=Visual Studio 17 2022" >> $env:GITHUB_ENV
188-
189-
- if: ${{ matrix.os == 'windows-2022' && matrix.arch == 'x64' }}
190-
shell: powershell
191-
run: echo "CMAKE_GENERATOR_PLATFORM=x64" >> $env:GITHUB_ENV
192-
193-
- if: ${{ matrix.os == 'windows-2022' && matrix.arch == 'x86' }}
194-
shell: powershell
195-
run: echo "CMAKE_GENERATOR_PLATFORM=Win32" >> $env:GITHUB_ENV
196-
197-
- if: ${{ matrix.os == 'windows-2022' && matrix.arch == 'arm64' }}
198-
shell: powershell
199-
run: echo "CMAKE_GENERATOR_PLATFORM=ARM64" >> $env:GITHUB_ENV
200-
201-
- if: ${{ startsWith(matrix.os, 'ubuntu-') }}
202-
name: Update apt sources
203-
run: sudo apt-get update
204-
205-
- if: ${{ matrix.platform == 'linux' }}
206-
name: Install required packages linux
207-
run: sudo apt-get install libxi-dev libxtst-dev libxrandr-dev
130+
- name: Setup Apple Clang
131+
uses: tetherto/qvac/.github/actions/setup-apple-clang@1d9b2165867d03c6edd675e402ee101a5d48a6d8
208132

209133
- name: Setup Vulkan SDK
210134
uses: tetherto/qvac/.github/actions/setup-vulkan-sdk@0bbdca93da303a0b1634ba14a89cec085621078d
@@ -216,23 +140,10 @@ jobs:
216140
aws-role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN }}
217141
aws-region: eu-central-1
218142

219-
- if: ${{ matrix.platform == 'android' }}
220-
name: Configure runner for cross compilation - android
221-
run: |
222-
echo "ANDROID_TOOLCHAIN_ROOT=$(echo $ANDROID_NDK_HOME)/toolchains/llvm/prebuilt/linux-x86_64" >> $GITHUB_ENV
223-
echo "ANDROID_NATIVE_API_LEVEL=34" >> $GITHUB_ENV
224-
225143
- name: Install npm dependencies
226144
working-directory: ${{ env.WORKDIR }}
227145
run: npm install
228146

229-
- if: ${{ matrix.platform == 'ios' || matrix.platform == 'darwin' }}
230-
name: Use Apple clang for Apple platform builds
231-
run: |
232-
sudo mv /opt/homebrew/opt/llvm@18 /opt/homebrew/opt/llvm@18.bak 2>/dev/null || true
233-
sudo mv /opt/homebrew/opt/llvm /opt/homebrew/opt/llvm.bak 2>/dev/null || true
234-
sudo rm -f /opt/homebrew/bin/clang /opt/homebrew/bin/clang++ 2>/dev/null || true
235-
236147
- name: Run bare-make generate
237148
working-directory: ${{ env.WORKDIR }}
238149
run: |
@@ -275,17 +186,10 @@ jobs:
275186
find prebuilds -maxdepth 1 -type d -print || true
276187
277188
- name: Strip debug symbols
278-
if: ${{ matrix.platform != 'win32' }}
279-
shell: bash
280-
working-directory: ${{ env.WORKDIR }}
281-
run: |
282-
if [[ "${{ matrix.platform }}" == "android" ]]; then
283-
find prebuilds \( -name "*.bare" -o -name "*.so" \) -exec $ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip {} \;
284-
elif [[ "${{ matrix.platform }}" == "darwin" || "${{ matrix.platform }}" == "ios" ]]; then
285-
find prebuilds -name "*.bare" -exec strip -S {} \;
286-
else
287-
find prebuilds -name "*.bare" -exec strip --strip-debug {} \;
288-
fi
189+
uses: tetherto/qvac/.github/actions/strip-prebuilds@1d9b2165867d03c6edd675e402ee101a5d48a6d8
190+
with:
191+
platform: ${{ matrix.platform }}
192+
workdir: ${{ env.WORKDIR }}
289193

290194
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # 7.0.0
291195
with:
@@ -309,4 +213,3 @@ jobs:
309213
with:
310214
name: prebuilds
311215
path: prebuilds
312-

0 commit comments

Comments
 (0)