Repository navigation
Conversation
|
This is a an x/tools/go/packages alternative to #1893, cc @ccoVeille. The benchmarks are the same as #1893 (comment). I did run into #1711 during verification, and I've run into it in projects using revive / golangci-lint. Would sure be nice to get one of the open fix PRs for that merged. |
5ac8a6c to
7a79f12
Compare
|
@alexandear just rebased this to get #1898. |
8223b53 to
a3ad8cc
Compare
|
Tried out this PR along with #1892 locally on my internal codebase. This PR enables finding many more #1892 fixes some Doing some more comparisons I am seeing bumps in time and memory. Checking those out. |
9b4e0db to
4b174b3
Compare
|
With the latest commit things now look like this for my internal codebase (145 packages, ~750 Go files and ~409k lines, and depends on ~880 non-stdlib packages). Warm build cache:
Empty cache (all rules):
Extra time is from producing export data for more things, as expected. (this comparison was done with #1892 merged in too for reproducibility) |
4b174b3 to
819fc78
Compare
|
@alexandear @ccoVeille anything else I can do to get this going? (and #1892 as well) |
ccoVeille
left a comment
There was a problem hiding this comment.
LGTM 👍
I asked a review to Copilot and it brought this
I skimmed the diff but didn't run it. I found no hard bug, only these concerns, roughly in order of importance:
1. **Errors are cached permanently.** `exportFile` stores failures in `m.exports`, and `moduleExportsCache` lives for the whole process. A transient failure, such as a `go list` or build hiccup, sticks for every later lookup. For a long-lived consumer, such as an LSP or library use, stale data is a problem too. The export files are build-cache paths that can be evicted or rebuilt, and the cached paths are never refreshed.
2. **`packages.Load` runs while holding `m.mu`.** This is correct, and it gives you the batching, but every goroutine in the same module blocks behind a slow `go list` or compile. The fallback path does one `packages.Load` per missing import path under the lock, so a package with many unresolved imports serialises badly.
3. **`batchErr` reaches only one caller.** `loadPending` clears `m.pending` before loading. If the batch fails, only the first path that misses gets `batchErr` in its error. Later misses run their own fallback load and may report a less informative error.
4. **`byDir` and the module-root cache assume a stable layout.** Resolvers are keyed by module root, so a later `Lint` call with a different `GOFLAGS`, build tags or environment reuses the old export data.
I'd raise 1 as the main concern, and 2 as a performance question.
Use packages.Load to resolve export data in the target module's build environment and importer.ForCompiler to load it for type checking. Load dependencies lazily in batches and cache results across packages. This fixes module import resolution and supports -trimpath builds. Add regression coverage for module and standard library imports. Closes revive-lint#1277
819fc78 to
2459019
Compare
|
Thanks! Changed that error flow. And changed things so each Lint call does its own caching instead of keeping things across calls. |
Use packages.Load to resolve export data in the target module's build environment and importer.ForCompiler to load it for type checking.
Load dependencies lazily in batches and cache results across packages. This fixes module import resolution and supports -trimpath builds.
Add regression coverage for module and standard library imports.
Closes #1277