Skip to content

Infer dependencies across local replace directives in the Go backend (closes #22097)#23431

Open
rdeknijf wants to merge 1 commit into
pantsbuild:mainfrom
rdeknijf:go-cross-module-replace-inference
Open

Infer dependencies across local replace directives in the Go backend (closes #22097)#23431
rdeknijf wants to merge 1 commit into
pantsbuild:mainfrom
rdeknijf:go-cross-module-replace-inference

Conversation

@rdeknijf

Copy link
Copy Markdown
Contributor

Disclaimer: Like my previous Go PR, I'm still not primarily a golang developer. However, the fact that Golang doesn't work properly in Pants has been the bane of my existence for like 2 years now. The moment that Mythos/Fable dropped I immediately had it dig deeply into whether the whole road to proper-golang-in-pants could be opened up. So this is all by Claude Code with Fable 5 (xhigh), with consults to GPT 5.5 (xhigh) and Gemini 3.1 Pro. I've had it check and recheck, I ran many different roles over it, and I had it explain and re-explain it to me, and then I checked myself.

So, as much as I dislike AI slop and am worried about AI PR overload, I've done my very best to avoid exactly that while still using AI. I hope we can get this road unblocked.

The rest is Fable talking:


Summary

Infer dependencies across local replace directives, so that a go_package that imports another first-party module wired in through a directory replace no longer needs its cross-module dependencies enumerated by hand.

This closes the TODO Issue #22097 that was sitting in third_party_pkg.py.

This is a correctness change, independent of my open Go performance PRs (#23420, #23421, #23422, #23424). It does not depend on any of them and can be reviewed on its own. For a monorepo that wires its first-party modules together with directory replace directives, it is the difference between dependency inference working and not working, so of my open Go changes this is the one I would prioritize for review.

Problem

A go.mod can replace a required module with a path to a local directory that holds another first-party module:

// context/app/go.mod
require example.com/shared v0.0.0
replace example.com/shared => ../shared

During third-party module analysis Pants skips these (they are not third-party packages), but nothing ever folded the replaced module's packages into the referencing module's import-path map. So an import of example.com/shared/pkg/field from context/app was never resolved by dependency inference, and the user had to list every cross-module dependency explicitly in BUILD metadata. In a repo where the entire first-party graph is wired together with directory replaces, that is most of the dependency edges.

Change

Three pieces, all in the Go backend:

  1. determine_go_mod_info now records each replace whose target is a local directory, mapping the replaced module path to that directory (relative to the build root). It reads this from the go mod JSON metadata Pants already fetches, so no extra process is run. Module-path replacements (a path plus a version) are left to normal third-party resolution. A small _is_directory_replacement helper mirrors Go's own modfile.IsDirectoryPath rule for what counts as a directory target.

  2. map_import_paths_to_packages folds the replaced modules' packages into the referencing module's import-path map. Because Go applies replace directives only within the module that declares them (they are not transitive), the merge is a single level deep. When a module has no directory replaces the rule returns the base mapping unchanged, so this is a no-op for the common case.

  3. The skip in analyze_module_dependencies and its stale TODO are updated to point at where the inference now happens.

The one subtlety worth calling out

When merging, only the replaced module's own packages are folded in, that is, import paths equal to or under its module path. Its third-party dependencies are deliberately not merged. The referencing module resolves those through its own go.mod, and if the shared third-party import paths were merged in too they would map to two addresses and become ambiguous, which silently suppresses the inferred dependency. The prefix filter is what keeps shared third-party imports (protobuf, grpc, cloud SDKs) inferring correctly. There is a dedicated test for exactly this.

Tests

Two new tests in target_type_rules_test.py:

  • test_cross_module_local_replace_dependency_inference: an import satisfied by a locally-replaced sibling module is inferred with no explicit dependency.
  • test_cross_module_local_replace_does_not_shadow_third_party: a third-party import shared by both modules still infers correctly and is not made ambiguous by the merge.

Full src/python/pants/backend/go/util_rules/:: and target_type_rules suites pass. Validated end to end on a real 24-module monorepo: every cross-module import that previously needed a hand-written dependency now infers, with no explicit dependencies, and the build is otherwise unchanged.

Security

No new process execution, no network access, no new filesystem reads. The change parses replace entries out of metadata Pants already collects and augments an in-memory inference map. Directory paths are normalized with os.path.normpath; absolute on-disk targets are skipped because they cannot map to an in-repo go_mod target. A replaced directory only contributes packages if it resolves to an existing go_mod target in the repo, so nothing outside the build graph is pulled in.

Compatibility

No behavior change for any module that does not use a directory replace: the merge path is skipped entirely when there are none. No options, no lockfile or cache-key changes.


Release note (add to docs/notes/2.33.x.md, under #### Go)

Dependency inference now resolves imports across local directory replace directives. When a go.mod replaces a required module with a path to another first-party module in the repo, imports of that module's packages are inferred automatically instead of having to be listed by hand in BUILD metadata. This is a no-op for modules without directory replace directives. See #22097.

…tsbuild#22097)

A `go.mod` may `replace` a required module with a local directory holding
another first-party module. Pants skipped such modules during third-party
analysis and never folded their packages into the referencing module's
import-path map, so imports of a locally-replaced module's packages were not
inferred; users had to enumerate every cross-module dependency explicitly in
BUILD metadata.

`determine_go_mod_info` now records local-directory `replace` targets, and
`map_import_paths_to_packages` merges those modules' import-path maps into the
referencing module's map. Go applies `replace` directives only within the
module that declares them, so the merge is one level deep.
@rdeknijf rdeknijf marked this pull request as ready for review June 16, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant