fix: release artifacts and verify them more #253
Merged
Merged
Conversation
dignifiedquire
changed the title
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
dignifiedquire
commented
Jun 25, 2026
…nnels Pre-publish verification, two layers per channel: - Layout checks (cheap, deterministic) — assert artifact shape. - Consumer smokes (real loads) — exercise the binding from a consumer perspective (iOS xcodebuild, Android emulator JNI, JNA from per-platform staged dir, fresh-venv wheel install). Wired into release.yml as publish gates and into pre-release-check for local runs. Catches the iroh-ffi#246 (missing platform slice) and #247 (wrong bundle layout) bug classes, plus 'lib in jar but won't load' regressions.
The release only cross-built desktop cdylibs (linux/macos/windows), so
computer.iroh:iroh shipped no libiroh_ffi.so for Android. Consumer APKs
built fine but crashed at runtime with UnsatisfiedLinkError the moment
IrohAndroid runs System.loadLibrary("iroh_ffi").
Add a build-kotlin-android job that cross-builds all four ABIs with
cargo-ndk, and stage them into the jar under lib/<abi>/, where AGP merges
lib/<abi>/*.so from JAR dependencies into the consumer APK.
Verified locally: AGP merges lib/<abi>/*.so from a plain jar into the
APK, and a real arm64-v8a build loads on an emulator — the hello-iroh-ffi
Android demo binds an endpoint and renders its id ("Ready") instead of
crashing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… smoke - ci_swift.yml: run verify-swift-xcframework + verify-swift-consumer after test-swift. The new verify-swift-consumer boots an iOS Simulator and runs IrohLibTests on the iOS runtime (not just a build smoke), catching link errors + iOS startup regressions that the macOS-only test misses. - ci_kotlin.yml: new kotlin-android-consumer-smoke job on ubuntu-latest with KVM-backed emulator. Mirrors release.yml's tag-only verify gate so every PR exercises the full Android consumer path (AGP merge from JAR → APK → emulator boot → JNI call), not just at release time.
New kotlin-jvm-consumer-smoke matrix in ci_kotlin.yml runs on linux x86_64, linux aarch64, macOS arm64, windows x86_64 — one runner per JNA platform the Maven jar ships. Each builds its host's release cdylib, stages it under the per-platform subdir, hides the dev-mode flat lib, and runs the gradle test suite. Catches 'JNA-platform name wrong' / 'loads only via flat fallback' on every PR for every shipped host. verify-kotlin-consumer now uses script_runner="bash" with a Windows case branch (uname returns MINGW64_NT-* in Git Bash) and a path-separator normalization for cargo metadata's backslash output on Windows. cp+mkdir replaces install -D -m644 to avoid relying on coreutils install on Git Bash (where its presence is busybox-implementation-dependent).
davidB/rust-cargo-make@v1 silently fails on linux-aarch64 self-hosted runners (downloads x86_64 binary only). taiki-e/install-action uses cargo-binstall to fetch the right precompiled binary for the host arch.
Without it, cargo-make on Windows writes the script to a .cmd file and runs it through cmd.exe, which choked on the bash case statement (esac literal token).
…, detect iOS SDK) Three orthogonal CI fixes for jobs added in this PR: - Windows JVM smoke: cargo-make's script_extension=sh didn't switch the shell on Windows; cmd.exe still parsed the bash script and choked on 'case'. Extract to scripts/verify_kotlin_consumer.sh and invoke via bash; @shell runner on cmd happily launches bash.exe (Git Bash is on PATH on the windows-latest image). - linux-aarch64 JVM smoke: taiki-e/install-action lacks a precompiled cargo-make for aarch64-linux and the cargo-binstall fallback errored on stale runner state. Replace with 'cargo install --locked cargo-make' guarded by 'command -v' so self-hosted runners reuse the binary between jobs. - Android emulator smoke: reactivecircus/android-emulator-runner runs each line of script: as a separate sh -c, so V=... + cd + ./gradlew lost state. Extract to scripts/run_android_smoke.sh. - verify-swift-consumer: add a precheck for iOS SDK presence so the failure is one clear error instead of buried-in-xcodebuild-output. The xcode16 self-hosted runner needs 'sudo xcodebuild -downloadPlatform iOS' run once; ci_swift.yml will go red until then.
The script form writes to a temp file with platform-default extension (.cmd on Windows) even when script_runner is set; cmd.exe then parses the bash body and dies on case/esac. command=bash + args=[script.sh] runs bash directly with no shell wrapper.
…smoke Two more fixes for the JVM consumer smoke matrix: - bindgen-kotlin (a dependency of verify-kotlin-consumer) had the same @shell + bash-script pattern that breaks on Windows. Extract to scripts/bindgen_kotlin.sh and switch to command=bash/args. - The self-hosted macOS-arm runner is running its agent under Rosetta: uname -sm reports 'Darwin x86_64' even though the hardware is M-series and the installed JVM is the arm64 build. The verify script was staging the cargo-default-host (x86_64) dylib under darwin-x86-64/, but the arm64 JVM's JNA looked at darwin-aarch64/ → UnsatisfiedLink. Use sysctl -n hw.machine on Darwin (immune to Rosetta) + explicit cargo --target per matrix entry. Matrix gains a cargo_target column passed to dtolnay/rust-toolchain so the target is installed before the script runs.
The unlabeled self-hosted macOS/ARM64 pool runs its agent under Rosetta, which broke arch matching for the JVM smoke. xcode16 is the known-native-arm pool (release runners use it too).
On windows-latest GitHub runners, bash.exe in PATH resolves to WSL's launcher (System32) — which fails with 'no installed distributions'. Git for Windows ships sh.exe but no equivalent bash conflict, so 'sh' reliably reaches Git Bash. Scripts are POSIX-compatible (no bashisms in the case patterns), validated with sh -n.
kotlin/lib's transitive net.java.dev.jna:jna:5.15.0 brings in the JVM JAR variant. The Android consumer needs the AAR variant (carries libjnidispatch.so per-ABI). AGP errors on both flavors in the classpath with 'Duplicate class com.sun.jna.*'. Exclude the transitive and pin the AAR explicitly.
kotlin/lib compiles with Kotlin 2.2.20 (per libs.versions.toml); the consumer at 2.0.21 rejected its 2.2.0 metadata. Bump Android module's Kotlin to 2.2.20, AGP to 8.13.0 (first version with K2.2 support), Gradle wrapper to 8.13 (AGP 8.13 requirement).
… -library
Replaces the hand-laid Iroh.framework skeleton (one .framework directory
per slice with a per-platform layout to maintain) with the Apple-blessed
flat library form: each slice ships lib<name>.a + Headers/ at the slice
root, no .framework subdirectory anywhere.
- make_swift.sh: assemble per-slice headers (uniffi's iroh_ffiFFI.h +
Export.h umbrella + module.modulemap declaring "module Iroh"), lipo
the two iOS sim arches together, then one xcodebuild
-create-xcframework invocation produces the whole xcframework with the
outer Info.plist auto-generated.
- Delete the committed Iroh.xcframework/ skeleton entirely — gitignore
the whole directory, treat it as a build artifact. xcframework is
regenerated from scratch on every build.
- Delete make_swift_macos.sh: docs-swift now depends on swift-xcframework
(the full 4-target build), so there's no separate macOS-only fast path
to maintain.
- verify-swift-xcframework: rewrite for the flat layout (slice has lib.a
+ Headers/{iroh_ffiFFI.h, Export.h, module.modulemap}; assert no
Iroh.framework/ leftover).
- Package.swift: local-built detection points at libiroh_ffi.a instead
of the .framework/Iroh binary path.
Removes the iroh-ffi#247 bug class entirely: with no .framework dir,
there's no deep-vs-shallow bundle layout for Xcode to reject. Apple
regenerates AvailableLibraries metadata against the current Xcode SDK
each build, so future Xcode majors don't need a hand-fix.
Validated locally:
cargo make swift-xcframework — OK (4-target build → flat layout)
cargo make verify-swift-xcframework — OK
cargo make test-swift — OK (xcodebuild test macOS; `import Iroh` resolves)
scripts/release/zip_xcframework.sh twice — same SHA-256
(same-host deterministic)
…ation Hardcoded `name=iPhone 16` worked on CI runners but not on developer machines that have e.g. iPhone 17 Pro / Air installed. `OS=latest` without a name doesn't resolve either. Query simctl for any available iOS sim and pass its UDID — locally and on CI, both find a valid destination.
…elease flow) The Swift xcframework was the only artifact this repo built twice — once locally during prepare-release to bake the SHA into Package.swift, once on tag-CI to upload the zip — with the two builds expected to produce byte-identical bytes. Cross-host determinism kept failing (Xcode patch versions, macOS SDK skew), forcing manual zip-clobber rituals post-publish. Single build, single source of truth: - release_swift.yml: new workflow that fires on push to release/v* branches (the convention prepare-release sets up). Builds xcframework + verifies layout + iOS sim test, computes SHA-256, commits the SHA into Package.swift on the PR branch (a bot commit marked [skip swift-release] so it doesn't re-fire), uploads the zip to a draft GitHub release v<version>. - release.yml's create-release: replaced actions/create-release@v1 (deprecated) with gh CLI that promotes the existing draft release to published — the swift zip baked in PR-CI is already attached. Falls through to fresh-create if no draft exists (manual tag without PR). - release.yml's build-and-publish-libs: switched actions/upload-release-asset to `gh release upload --clobber` since the new create-release no longer produces an upload_url output. - release.yml's build-and-publish-swift: deleted. The swift zip ships with the promoted draft. - prepare-release: stripped the xcframework build + checksum bake. No Xcode required locally anymore — versions-only edit, PR-CI fills the SHA when you push.
…w flow - pre-release-check: clarify that swift xcframework is built here as a sanity check only; the shipping zip comes from release_swift.yml on PR CI. Same-host determinism check kept (catches stale caches). - RELEASING.md: rewrite for the two-phase version bump. Local prepare-release does versions only (no Xcode required); PR CI's release_swift.yml bakes the Swift xcframework SHA into Package.swift on the release branch and uploads the zip to a draft GitHub release. Tag CI just promotes the draft. New 'Why PR-CI builds the Swift xcframework' section documents the determinism trade-off the old local-baked flow kept hitting. - Package.swift, bump_version.py, zip_xcframework.sh: comments updated to match the new flow (releaseChecksum is written by PR CI, not local; zip_xcframework's determinism is now same-host sanity, not cross-host reproducibility).
… arch detect sysctl -n hw.machine and uname -m both report the *process* arch, not the hardware — so on Apple Silicon under Rosetta, both return x86_64 and the script stages an x86_64 dylib that the native-arm64 JVM can't load (18 JNA tests fail). hw.optional.arm64 returns "1" on Apple Silicon hardware regardless of process arch, which is the actual question.
Stops bundling Android-only types + native libs into the JVM-only computer.iroh:iroh JAR. Android consumers will depend on a new computer.iroh:iroh-android AAR (D2 stages the .so files into the AAR's jni/<abi>/ path; D3 wires the consumer smoke to the new coordinate). - Gradle 9.0 -> 8.13: AGP 8.13 is the latest stable that supports Kotlin 2.2.x, and it requires Gradle 8.13. All other plugins (kotlin-jvm 2.2.20, dokka 2.0, vanniktech-publish 0.34) work on 8.13. - New kotlin/build.gradle.kts at the root declares every plugin with apply false so subprojects share one classloader per plugin. Without it Vanniktech's MavenCentralBuildService loads twice (once per subproject) and Gradle errors on the conflicting plugin instances. - New kotlin/android/ subproject (com.android.library + kotlin-android + vanniktech AndroidSingleVariantLibrary). Publishes computer.iroh:iroh -android, has IrohAndroid moved over from :lib, api(:lib) so Android consumers transitively get the JVM Kotlin API. JNA AAR explicit (with the JAR excluded from :lib transitive) to avoid duplicate-class. - :lib loses compileOnly(com.google.android:android:4.1.1.4) since IrohAndroid is no longer there. - bump_version.py rewrites coordinates in both kotlin/lib/build.gradle.kts and kotlin/android/build.gradle.kts so the two artifact versions stay in lockstep. Validated locally with `./gradlew :lib:tasks` and `:android:tasks` — both BUILD SUCCESSFUL. D2 will move the Android native staging path from kotlin/lib/src/main/resources/lib/<abi>/ to kotlin/android/src/main/jniLibs/<abi>/.
AGP requires the Android SDK at *configuration* time, so Gradle errors with 'SDK location not found' on jobs that previously didn't need it (JVM test, JVM consumer smoke matrix, build-and-publish-kotlin) because they now have to configure the :android subproject alongside :lib.
D2 — Android native staging path moves from
kotlin/lib/src/main/resources/lib/<abi>/ (JVM JAR's lib/<abi>/)
to
kotlin/android/src/main/jniLibs/<abi>/ (AAR source set; AGP
packages into the AAR's
jni/<abi>/, then merged
into consumer APKs).
Updated:
- cargo make kotlin-android task (writes to jniLibs path).
- release.yml stage-natives step (writes both JVM resources + AAR
jniLibs from the cdylib + android-jniLibs artifacts).
- ci_kotlin.yml kotlin-android-consumer-smoke job (same).
- verify-kotlin-artifact (validates both source trees instead of
just kotlin/lib/src/main/resources).
- publish step: ./gradlew publishAndReleaseToMavenCentral now (no
:lib: prefix) so both subprojects publish.
- .gitignore: kotlin/android/src/main/jniLibs/ is CI-generated.
D3 — kotlin/android-smoke consumes the new AAR coordinate. Composite
build substitutes computer.iroh:iroh-android → :android (was iroh →
:lib). app/build.gradle.kts depends on iroh-android (which transitively
brings in :lib + JNA AAR correctly).
Also: make :android subproject conditionally included (skip on hosts
without an Android SDK, e.g. the self-hosted linux-aarch64 runner
where setup-android@v3 fails). Lets :lib build + test on that runner
without the SDK. JVM smoke + JVM test jobs skip setup-android on
linux-aarch64.
- Workflow table cell now names both artifacts. - New 'Kotlin: two artifacts, one publish' section explaining the JAR vs AAR split and the one-line dep migration for Android consumers.
…orted Gradle 8.13 + AGP 8.13 + Kotlin 2.2.20 require Java 17-21. Many macOS devs have Homebrew openjdk (currently 26) as their default JAVA_HOME; gradle dies with a one-line '26.0.1' error message that doesn't say why. Detect the situation via /usr/libexec/java_home -v 17 and override locally. CI is unaffected (setup-java@v4 sets JAVA_HOME=21 explicitly before the task runs). Also scope test-kotlin to :lib:test specifically rather than the root 'test' alias — the :android subproject's empty test task would configure unnecessarily.
…se_swift shell
Five review threads:
- bump_version.py docstring: make it explicit that --checksum returns
without bumping any version literals; that path is purely for
release_swift.yml on PR CI.
- kotlin/android-smoke: was a separate Gradle root with its own wrapper
(8 binary + script files checked in). Convert to a subproject of the
kotlin/ root — share the parent wrapper. depends on :android via
project(":android"), composite-build substitution gone. delete the
duplicate gradle wrapper jar + gradlew + settings.gradle.kts +
gradle.properties + .gitignore. CI workflows + Makefile.toml +
scripts/run_android_smoke.sh now invoke
./gradlew :android-smoke:connectedDebugAndroidTest from kotlin/.
- ci_kotlin.yml comments trimmed: most one-liners or removed (the
reasoning lives in commit history). yaml ~30% shorter.
- release_swift.yml inline bash → scripts/release/swift_bake_sha.sh and
swift_upload_draft.sh. Both reusable for local dry-runs:
cargo make swift-xcframework && bash scripts/release/zip_xcframework.sh
bash scripts/release/swift_bake_sha.sh
bash scripts/release/swift_upload_draft.sh
- bonus: extract scripts/stage_android_natives.sh (shared by
ci_kotlin.yml's smoke and release.yml's verify) and
scripts/release/stage_kotlin_publish_natives.sh (release.yml publish
staging). yaml files now mostly thin orchestration around scripts.
When :android-smoke was a separate Gradle root it had its own kotlin/android-smoke/gradle.properties with android.useAndroidX=true. Flattening into a subproject lost that file; the consolidated root kotlin/gradle.properties didn't yet carry the AGP requirements.
dignifiedquire
force-pushed
the
ci/verify-release-artifacts
branch
from
5b05782 to
09f1099
Compare
|
ty for the ping! Build via Xcode fails for me on both iPhone and Mac targets with two FFI errors:
Package version: 82754a1 |
Contributor
|
@iLynxcat sorry I didn't get that error? |
|
strange - I attempted to clear DerivedData and build caches several times to ensure it wasn't a cache issue and I still got it. Will take more of a look soon, ty for the heads up. |
Member
These functions were introduced in #257 so it seems like this must be related to recent changes? Or maybe it is some build cache issue? |
Contributor
Author
|
@iLynxcat how did you load it, did you run `cargo make swift-xcframework? that is needed to generate the latest version, otherwise it will pick up the published version likely |
|
I tried this morning in a fresh Xcode project, fresh DerivedData, fresh cargo registry, and ran into the same two errors. I did cd into the package checkout path to run I do think it was pulling the release version of iroh itself, this build error seems like a separate issue. Sorry that I don't have much time to look into this further atm, I hope this helps point us in the right direction |
Contributor
Author
|
thanks @iLynxcat gonna merge this anyway, and hopefully once it is published it will be fixed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consumes #251, #246
Closes #247