aws_sdk_vendor: fetch aws-sdk-cpp live via ament_vendor(), drop the separate vendor repo - #437
Merged
Minipada merged 2 commits intoAug 30, 2026
Conversation
…ct flattened source ROS buildfarm binarydeb jobs actually run with `docker run --net=host` (ros_buildfarm's own job-generation source), and zmqpp_vendor proves the live-git-clone-at-build-time idiom already works there. That invalidates the no-network-access assumption #425/PR #433's flattened-source-plus-own-repo design rested on, so this reverts to a live fetch instead — now via ament_cmake_vendor_package's ament_vendor() macro (VCS_TYPE git) rather than a hand-rolled ExternalProject_Add, matching zmqpp_vendor/tinyxml_vendor/ yaml_cpp_vendor. GLOBAL_HOOK is required since dc_bridge only ever find_package(AWSSDK)s, never find_package(aws_sdk_vendor) first. Verified end to end: a from-scratch `colcon build --packages-select aws_sdk_vendor` succeeds (13min), `colcon build`/`colcon test` for the full workspace pass (634 tests, 0 failures), and `ldd` against the built dc_bridge binary resolves every AWS SDK/CRT library. The network-isolation check still fails under --network=none, now permanently by design (docs/adr/0012), not as a TODO pending #425. Also archives the github.com/Minipada/aws_sdk_vendor repo #425/PR #433 created — nothing in this workspace pulls from it. Closes #434 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169YY4yK42mPfah3DbsXwmt Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## jazzy #437 +/- ##
=======================================
Coverage 70.93% 70.93%
=======================================
Files 119 119
Lines 7401 7401
=======================================
Hits 5249 5249
Misses 2152 2152
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The previous commit archived github.com/Minipada/aws_sdk_vendor and dropped aws_sdk_vendor into this repo as a local package. Per feedback, that's the wrong call: this repo's vendor packages live in their own repos as a rule (matching vector_vendor, ADR-0002's amendment), independent of whether a given package's own content is large enough to justify the split on git-bloat grounds alone — a thin ament_vendor() recipe carries no such bloat, but splitting it anyway keeps every vendor package's layout and bump/release workflow consistent. - Un-archives github.com/Minipada/aws_sdk_vendor, replaces its flattened vendored source with the same live-fetch ament_vendor() recipe this repo briefly held locally (now also supporting an AWS_SDK_BUILD_ONLY override for downstream consumers needing services other than s3), and gives it its own CI (Podman build + prek format check). - Removes the local aws_sdk_vendor/ package from this repo; adds it to ros2_data_collection.repos (vcs import), the same path vector_vendor uses. - tools/e2e/Containerfile no longer COPYs a local aws_sdk_vendor/ directory — it arrives via toolchain-base's existing vcs import step alongside vector_vendor. - docs/adr/0012 rewritten to record the "own repo" decision and its rationale; doc/src/dc/setup.md updated for the new .repos entry. Re-verified end to end after the change: colcon build --packages-select aws_sdk_vendor succeeds from the new repo (13min), the full workspace build + colcon test pass (634 tests, 0 failures), and ldd against the built dc_bridge binary resolves every AWS SDK/CRT library. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169YY4yK42mPfah3DbsXwmt Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
github-actions
Bot
deleted the
feature/434-aws-sdk-vendor-fetch-aws-sdk-cpp-live-fr
branch
August 30, 2026 11:12
This was referenced Aug 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aws_sdk_vendorfetches aws-sdk-cpp directly fromhttps://github.com/aws/aws-sdk-cpp.gitat the pinned tag viaament_cmake_vendor_package'sament_vendor()macro (VCS_TYPE git), instead of a hand-rolledExternalProject_Add— the same idiomzmqpp_vendor/tinyxml_vendor/yaml_cpp_vendoralready use successfully on the real ROS buildfarm. No aws-sdk-cpp source is vendored anywhere.ros_buildfarm's own job-generation source runs binarydeb builds withdocker run --net=host, andzmqpp_vendorhas a real, currently-succeeding Jenkins job doing exactly this live-clone pattern.crt/aws-crt-cpp's submodule chain (including the unrelated CBMC formal-verification model) is now let recurse in full viaament_vendor()'svcs import --recursive, dropping the previous hand-rolledPATCH_COMMANDexclusion hack — harmless with real network available.GLOBAL_HOOKis required on theament_vendor()call:dc_bridgeonly ever callsfind_package(AWSSDK), neverfind_package(aws_sdk_vendor)first, so the install prefix has to reachCMAKE_PREFIX_PATHvia an environment hook, not a CMake config-extra.aws_sdk_vendorkeeps its own separate repo (github.com/Minipada/aws_sdk_vendor, mirroringvector_vendor's split, ADR-0002's amendment) — this repo's vendor packages live in their own repos as a rule, independent of whether a given package's own content (a thinCMakeLists.txt+package.xmlhere, no vendored source) is large enough to justify the split on git-bloat grounds alone.ros2_data_collection.reposgains anaws_sdk_vendorentry, the samevcs importpathvector_vendoralready uses. vector_vendor: fetch the Vector release tarball live via checksum-pinned download, reconsider its separate repo #435 records the same decision forvector_vendor.aws_sdk_vendor/CMakeLists.txtnow also exposes anAWS_SDK_BUILD_ONLYCMake cache variable (defaults3, all this workspace needs) so a downstream project can build other AWS service clients without forking the package — forwarded to aws-sdk-cpp's own-DBUILD_ONLYvia a$<SEMICOLON>-escaped generator expression (verified: a plain quoted value silently mis-splits a multi-service list before it reachesExternalProject_Add). It also gets its own CI — a PodmanContainerfilebuild plus aprek-driven format check — mirroring this repo's own conventions.docs/adr/0012-aws-sdk-vendor-flattened-source.mddocuments the buildfarm network-access finding and both decisions (live fetch, own repo).tools/e2e/Containerfile,.github/workflows/ci.yaml, andtools/e2e/scripts/verify_network_isolation.shcomments updated: the network-isolation-check job is now documented as a permanent, by-design failure foraws_sdk_vendor(not a TODO pending aws_sdk_vendor: flattened source, no network fetch #425), and no longerCOPYs a localaws_sdk_vendor/directory since it now arrives viatoolchain-base's existingvcs importstep (alongsidevector_vendor)..pre-commit-config.yamlandREUSE.tomlneeded no changes.github.com/Minipada/aws_sdk_vendorrepo (created for aws_sdk_vendor: flattened source, no network fetch #425/PR feat(aws_sdk_vendor): flattened source, no network fetch #433's now-superseded flattened-source design) is un-archived and its content replaced with the thin wrapper above — not deleted, since it's needed again.Verification (acceptance criteria)
colcon build --packages-select aws_sdk_vendorfrom scratch, fetched from the satellite repo, succeeds — ~13min, clean except one benign upstreamLEGACY_BUILDCMake warning.colcon build+colcon test(tools/e2e/Containerfile'sworkspacestage): 17 packages build, 634 tests pass, 0 failures,dc_bridgebuilds and links in ~15s.lddagainst the builtdc_bridgebinary (after sourcinginstall/setup.bash) resolves every AWS SDK/CRT.so— no "not found" entries.tools/e2e/scripts/verify_network_isolation.shstill fails as designed:aws_sdk_vendor's build fails under--network=none(the download step errors out) whilevector_vendor's succeeds — confirming this is a permanent, attributable seam, not a regression.AWS_SDK_BUILD_ONLYescaping was verified in isolation (a standaloneExternalProject_Addharness) for both the default single-value case and a multi-service (s3;dynamodb) override, confirming the inner configure step receives the correct semicolon-separated list either way.github.com/Minipada/aws_sdk_vendor's own new CI (Podman build) verified locally against itsContainerfile.prek run --all-files --skip build-docpasses clean in this repo, includingreuseandhadolint;reuse lintpasses clean in the satellite repo too.Closes #434
🤖 Generated with Claude Code
https://claude.ai/code/session_0169YY4yK42mPfah3DbsXwmt