[WIP] bazel: Add LLVM toolchain

[phlax]

Ideally we use this toolchain for compiling and can then remove the compiler from our container platform/toolchains

More immediately, adding this means we can use the clang tools (clang-format, clang-tidy, llvm-cov, etc) with bazel - this remove the host requirement, and should prevent issues around running tooling with incorrect versions (in combination with #29397 and some further dev)

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

[repokitteh-read-only]

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).
envoyproxy/dependency-shepherds assignee is @moderation

🐱

Caused by: #29415 was opened by phlax.

see: more, trace.

[phlax]

@keith i had to set the version to 14.0.0 (our current) as there is not a toolchain for ubuntu < 22 - but that should be ok for now

i also had to shift some of the setup around to get it working, not exactly ideally

[phlax]

hmm - so the problem with this plan is that the llvm repo/tarball still moves around as a 4GB block where its invoked locally

for the most part CI can run without invoking any of the bins locally, but if eg you try to use the (100MB) clang-tidy or clang-format bins it pulls the full 4GB - this makes linting have a much bigger overhead than compiling

in the cases of clang-tidy/format specifically there are python packages that you can use to just pull the bins and nothing else

another problem is that the binary versions dont seem to be well maintained - checking from llvm release to the next there are not the same set of assets. Fwiw the python (invoked) bins mentioned above suffer from similar issue with sporadic releases

The version problem is already there - whatever distribution method we use we are relying on prebuilt binaries that work on ubuntu 20.04 - the current version may be the last one that these are built for

cc @keith

[keith]

there are some llvm threads about improving the release process for these binaries so I'm optimistic that part will get solved in the future. not sure what to say about the binary size problem though. That was always something we were going to have to take the hit on if we moved to this. If this was the default then we could probably stomach doing that download once since you would keep the binaries locally until they were upgraded again anyways

[phlax]

That was always something we were going to have to take the hit on if we moved to this.

one consequence is that we cant use this without removing from the container image otherwise ci downoads 4G x2

If this was the default then we could probably stomach doing that download once since you would keep the binaries locally until they were upgraded again anyways

its more about CI and limiting what an RBE client needs to download on a run (esp where there is no change)

for the compiler i agree we just need to take the hit

for clang tools im not so sure - for now we can hook up the python way as it ~matches our current version

upgrading tho we will then need an upstream to be available for both the full compiler toolchain and any tooling

as we are in this situation anyway (no current upgrade path) im happy to land this and start using the python distributed tools

its gonna be a bit fragile but not much worse than current

the only other blocker i hit was that i got a compiler error when trying to use for compile toolchain - ill re-enable it so we can see

[phlax]

#29455

[github-actions]

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

[phlax]

this is probably worth another try

the llvm_toolchain has moved to bazel-contrib

there is also a potential gcc toolchain here https://github.com/f0rmiga/gcc-toolchain

[mathetake]

I like this - this also will make upgrades of LLVM much easier