Skip to content

Commit f6a3106

Browse files
committed
Ruby: Change how we pull in shared/tree-sitter-extractor dependency
Previously, we pulled in the shared tree-sitter extractor via a `git` dependency in `Cargo.toml` to address a `rules_rust` limitation (no `path` dependencies outside of the cargo workspace)). This was a problem, as that means we're cloning `github/codeql` _again_ for the build, which is quite slow. I found another way that is faster, and still produces correct builds for both `cargo`` and `rules_rust`: * Cargo depends on a fake crate that has the same dependencies as the real crate (thanks to `sync-files.py`). Therefore, cargo pulls in the right dependencies into the lockfile, which bazel targets * For local builds, we override the path to that dependency in a cargo config, so we're pulling in the correct code * rules_rust only uses `path` dependencies for collecting transitive dependencies, it never pulls in the code from there. So far that, we manually provide a `BUILD.bazel` file for the shared extractor, and depend on that.
1 parent 0dbce3d commit f6a3106

File tree

11 files changed

+666
-44
lines changed

11 files changed

+666
-44
lines changed

config/identical-files.json

+4
Original file line numberDiff line numberDiff line change
@@ -364,5 +364,9 @@
364364
"Python model summaries test extension": [
365365
"python/ql/test/library-tests/dataflow/model-summaries/InlineTaintTest.ext.yml",
366366
"python/ql/test/library-tests/dataflow/model-summaries/NormalDataflowTest.ext.yml"
367+
],
368+
"shared tree-sitter extractor cargo.toml": [
369+
"shared/tree-sitter-extractor/Cargo.toml",
370+
"ruby/extractor/codeql-extractor-fake-crate/Cargo.toml"
367371
]
368372
}

ruby/extractor/.cargo/config.toml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
paths = ["../../shared/tree-sitter-extractor"]

ruby/extractor/BUILD.bazel

+3-1
Original file line numberDiff line numberDiff line change
@@ -11,5 +11,7 @@ codeql_rust_binary(
1111
visibility = ["//visibility:public"],
1212
deps = all_crate_deps(
1313
normal = True,
14-
),
14+
) + [
15+
"//shared/tree-sitter-extractor:codeql-extractor",
16+
],
1517
)

ruby/extractor/Cargo.lock

+76-4
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

ruby/extractor/Cargo.toml

+4-9
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
[workspace]
12
[package]
23
name = "codeql-extractor-ruby"
34
description = "CodeQL Ruby extractor"
@@ -27,14 +28,8 @@ encoding = "0.2"
2728
lazy_static = "1.4.0"
2829
# Ideally, we'd like to pull this in via a relative path.
2930
# However, our bazel/rust tooling chokes on this, c.f. https://github.com/bazelbuild/rules_rust/issues/1525
30-
# Therefore, to break that dependency, we depend on it via a git dependency instead.
31-
# We should change this back to a path dependency once this issue is fixed.
32-
# We can't depend on this without a rev/branch specification, as the rules_rust code assumes the default branch
33-
# is called `master`, and if we pull this in with `branch=main`, then `cargo` works (and pins this at th current git SHA
34-
# of lock-file update time, but `rules_rust` pins generates a bazel rule that unconditionally downloads `main`, which
35-
# breaks build hermeticity. So, rev-pinning it is.
36-
# See also https://github.com/bazelbuild/rules_rust/issues/2502.
37-
codeql-extractor = { git = "https://github.com/github/codeql.git", rev = "a523be4d0a1e2420a1884f7c4f8754a7c4fb7e21" }
31+
# Therefore, we have a pretty bad hack in place instead, see README.md in the codeql-extractor-fake-crate directory.
32+
codeql-extractor = { path = "codeql-extractor-fake-crate" }
3833

3934
[patch.crates-io]
40-
tree-sitter = {git = "https://github.com/redsun82/tree-sitter.git", rev = "1f5c1112ceaa8fc6aff61d1852690407670d2a96"}
35+
tree-sitter = { git = "https://github.com/redsun82/tree-sitter.git", rev = "1f5c1112ceaa8fc6aff61d1852690407670d2a96" }

0 commit comments

Comments
 (0)