BuildBuddy GitHub app: Submodules

[ivucica]

Hello!

I have a toy project repo where I've quasi-vendored a few other rules repos by adding them as Git submodules, and then using WORKSPACE rules to pull them in via local_repository().

These are perfectly usable with GitHub Actions, but if I get the BuildBuddy GH app to run bazelisk et al, it doesn't check out the submodules first. I haven't spotted a way to make it do so.

While I could by now switch over to pulling them in via a HTTP archive (perhaps write a wrapper repo rule that falls back to that approach if the submodule is not checked out, or write a wrapper rule that runs git submodule update --init if the submodule is not checked out), it would be a useful feature in BuildBuddy as well.

[sluongng]

I would recommend using git_repository instead of local_repository and git submodule as Bazel will lazily fetch the git repositories only when they are needed for the build.

Our BuildBuddy Workflows should work with git_repository today. There is a small bug reported this week that would prevent using git_repository on private repos, we are actively working on a fix for this.

@ivucica could you switch from using git submodule to git_repository?

[ivucica]

At this time, I have a strong preference to keep this project on submodules: editing the dependency is significantly easier with submodules than when leveraging git_repository. I might make it use one or the other depending on where the build is happening (CI vs not-CI), but for dev work, submodules are significantly nicer for turnaround.

(I recently had a similar irritation with Go modules: using replace is siginficantly worse than using GOPATH.)

[sluongng]

@ivucica thanks for replying!

It's possible to checkout git submodule in our Workflows offering today, but you would have to create a sh_binary that calls git submodule update --init and call it with bazel run at the first step in your BuildBuddy Workflows. We added a credential helper to the current repository, so the subsequent git fetch that git submodules call underneath should all have the correct credentials included.

I will discuss with my team to see if we could add this as an opt-in config for BuildBuddy Workflows. 🤔

[ivucica]

Thanks -- that's a good point; I suppose as long as it's run through Bazel, I can run anything.

FWIW checking out submodules became a thing in GitHub Actions embedded into the "stock" @actions/checkout at least as of v4, so if I partially run the bazel build workflow on GitHub Actions (meaning some secrets wrangling is in order), all is good:

https://github.com/opentibia/yatc/blob/9c74d6face56b28d8f2ab5ac64fe47867a400fac/.github/workflows/ci-bazel.yml#L61-L63

[ivucica]

That unfortunately does not do the trick because WORKSPACE-based repo rules are seemingly evaluated before target evaluation done in build and run can choose that it does not actually need a particular repo.

ERROR: /home/buildbuddy/workspace/repo-root/WORKSPACE:6:17: 
    fetching local_repository rule //external:rules_OMITTED: java.io.IOException: No WORKSPACE file
    found in /home/buildbuddy/workspace/output-base/external/rules_OMITTED
ERROR: Error computing the main repository mapping: no such package '@rules_OMITTED//': No WORKSPACE file found in /home/buildbuddy/workspace/output-base/external/rules_OMITTED