Skip to content

vector_vendor: no network fetch, split into its own repo - #432

Merged
Minipada merged 1 commit into
jazzyfrom
feature/vector-vendor-no-network-fetch
Aug 28, 2026
Merged

vector_vendor: no network fetch, split into its own repo#432
Minipada merged 1 commit into
jazzyfrom
feature/vector-vendor-no-network-fetch

Conversation

@Minipada

Copy link
Copy Markdown
Owner

Summary

Supersedes #429 and #431, consolidated into one PR: those two stacked PRs would have had jazzy gain the checked-in binaries in #429's commit and then delete them again in #431's, via a rebase merge that lands each commit individually — meaning the ~106MB binary actually writes into jazzy's permanent history either way, just hidden from the final tree diff. This PR goes straight from the old file(DOWNLOAD ...) state to the final split state in one commit; the binary tarballs never touch ros2_data_collection's history at all.

  • vector_vendor's file(DOWNLOAD ...) network fetch is gone. Fixing that in-place would mean checking the binary into this repo, and git never deduplicates that ~106MB across future Vector version bumps.
  • Instead, vector_vendor moves entirely into its own repo: https://github.com/Minipada/vector_vendor (default branch jazzy, tagged v0.57.0). Same CMakeLists.txt logic (checked-in tarball, checksum-pinned, no network at build time) — just relocated.
  • Pulled into this workspace via ros2_data_collection.repos (vcstool), pinned to the tag, not a branch, for reproducibility.
  • tools/e2e/Containerfile's toolchain-base stage runs the vcs import (network required, same footing as the rosdep install right below it); every downstream colcon build — including Network-isolated build verification harness for vendor packages #423's --network=none check — builds against the already-fetched result.
  • dc_bridge needs no changes: it locates the installed binary via the fixed lib/vector_vendor/vector path and an <exec_depend>, both unaffected by which repo the source came from.
  • ADR-0002 amended with the full reasoning, including considered-and-rejected alternatives: Git LFS (verified it would survive bloom's release export, by tracing and testing the actual bloom/vcstools code — rejected anyway on GitHub's free LFS quota math, not on that original, now-disproven assumption), compiling Vector from source (cargo vendor comes to ~1.6GB of dependency source, worse than the ~106MB binary), and a C++ shipper alternative (none with Vector's feature set exists).
  • Updated doc/src/dc/setup.md, four tools/e2e/scripts/*.sh that read VECTOR_VERSION out of the now-removed vector_vendor/CMakeLists.txt (now read from ros2_data_collection.repos instead), and stale CI/doc comments.

Test plan

  • prek run --all-files --skip build-doc passes
  • ./tools/e2e/scripts/verify_network_isolation.sh: vendor-network-check builds vector_vendor under --network=none, fetched via the external repo through toolchain-base's vcs import
  • Full production build (colcon build + all 634 tests) succeeds with vector_vendor fetched externally — verified locally and, for the underlying mechanism, on real CI infrastructure (a manually-triggered run against this work before consolidation)
  • Confirmed via git rev-list --objects that this branch's commit contains zero binary blobs — the squash from the prior two-PR attempt correctly netted them out entirely

Closes #424

🤖 Generated with Claude Code

https://claude.ai/code/session_019hV3DGw8o1QKsYbo8fQaS9

vector_vendor fetched its pinned Vector binary over the network at build time
via file(DOWNLOAD ...), which the ROS buildfarm's network-isolated binarydeb
jobs can't do (#423's network-isolated check proves it). Fixing that in-place
would mean committing the binary (~106MB for both architectures) into this
repo, which git never deduplicates across commits -- every future Vector
version bump would permanently grow ros2_data_collection's own history by
that much again, accepted and then rejected via Git LFS (evaluated: LFS
content does survive bloom's release-tarball export step, contrary to initial
assumption -- but GitHub's free LFS tier is 1GB/month, and ~106MB/bump would
exhaust it in roughly 9 bumps).

Instead, vector_vendor moves into its own repo entirely:
github.com/Minipada/vector_vendor (default branch jazzy, tagged v0.57.0).
Same CMakeLists.txt logic either way -- checked-in tarball, checksum-pinned,
no network at build time -- just relocated so the binary's growth lands
there instead of here. Pulled into this workspace via
ros2_data_collection.repos (vcstool), pinned to the tag (not a branch) for
reproducibility. tools/e2e/Containerfile's toolchain-base stage runs the
`vcs import` (network required, same footing as the rosdep install right
below it); every downstream colcon build -- including #423's --network=none
check -- builds against the already-fetched result, unaffected by where the
source physically came from.

dc_bridge needs no changes: it locates the installed binary at
lib/vector_vendor/vector (vector_vendor's own install path, unchanged) and
declares <exec_depend>vector_vendor</exec_depend> in package.xml -- both
resolve identically regardless of which repo the source came from, since
colcon discovers packages by scanning src/ for package.xml, not by git
origin.

Verified directly, not assumed: the vendor-network-check stage builds
vector_vendor under --network=none (both locally and on real CI
infrastructure via a manually-triggered workflow_dispatch run); a full
workspace build (colcon build + all 634 tests) succeeds with vector_vendor
fetched from the external repo; the checked-in-binary-vs-LFS tradeoff was
traced through the actual bloom/vcstools release code and tested against a
throwaway LFS repo, not just reasoned about.

This is a development-workspace convenience only (.repos/vcstool); it
doesn't make vector_vendor apt-installable on its own -- that needs an
independent bloom-release, not done here. See ADR-0002's amendment for the
full reasoning and the considered-and-rejected alternatives (Git LFS,
compiling Vector from source via `cargo vendor`, a C++ shipper alternative).

Closes #424

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019hV3DGw8o1QKsYbo8fQaS9
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.93%. Comparing base (6403634) to head (7183446).

Additional details and impacted files
@@           Coverage Diff           @@
##            jazzy     #432   +/-   ##
=======================================
  Coverage   70.93%   70.93%           
=======================================
  Files         119      119           
  Lines        7401     7401           
=======================================
  Hits         5249     5249           
  Misses       2152     2152           
Flag Coverage Δ
cpp-jazzy 70.93% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Minipada
Minipada merged commit 71be58b into jazzy Aug 28, 2026
18 of 19 checks passed
@github-actions
github-actions Bot deleted the feature/vector-vendor-no-network-fetch branch August 28, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant