Skip to content

fix: resolve typed-rule imports via go list export data - #1893

Closed
danp wants to merge 1 commit into
revive-lint:masterfrom
danp:issue-1277
Closed

danp wants to merge 1 commit into
revive-lint:masterfrom
danp:issue-1277

Conversation

@danp

@danp danp commented Oct 9, 2026

Copy link
Copy Markdown

importer.Default() only resolved stdlib imports, and none at all when revive was built with -trimpath, so typed rules silently missed most issues.

Resolve imports with importer.ForCompiler and a lookup backed by one lazy, cached go list -export batch per module. Results no longer depend on GOROOT, GOPATH or -trimpath.

Closes #1277

importer.Default() only resolved stdlib imports, and none at all when
revive was built with -trimpath, so typed rules silently missed most
issues.

Resolve imports with importer.ForCompiler and a lookup backed by one
lazy, cached `go list -export` batch per module. Results no longer
depend on GOROOT, GOPATH or -trimpath.

Fixes revive-lint#1277
@danp

danp commented Oct 9, 2026

Copy link
Copy Markdown
Author

Trying this based on @ccoVeille's suggestion to use importer.ForCompiler with our own lookup. It uses the go command so it's less to do, is module-aware, and independent of GOROOT/GOPATH.

Target Version Findings Cold build cache Warm
fiber ./... (unhandled-error) master 63 19.7s 1.83s
this PR 155 8.7s 0.64s
revive ./... (7 typed rules) master 16 8.8s 0.81s
this PR 25 6.4s 0.38s

Results with and without -trimpath are identical.

Build cache grows a bit more (~2x on fiber) if not already filled. And it does run the go command of course.

Reproductions against upstream fiber since the fork in #1277 (comment) doesn't seem to exist anymore.

@ccoVeille

ccoVeille commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

I’m happy you worked on this, the approach is interesting.

But, I was surprised to see the go binary being invoked directly here. That means revive now depends on a local Go installation being present, and potentially on it matching the Go version used to build revive. If those differ, I’m not sure how reliable this will be in practice.

I asked Copilot for feedback, and it basically confirmed my concerns, but also explain the possible solutions

**Your concern is valid.** The new `lint/importer.go` runs `exec.Command("go", ...)` with `go list -deps -test` and then `go list -export`. That has several consequences:
- The `go` binary has to be on `PATH`, and its version is whatever is installed. Export data from that toolchain must be readable by the `go/importer` in the Go version revive was built with. If the two differ, `importer.ForCompiler("gc", ...)` can fail on a format mismatch.
- It compiles dependencies, so it is slow on a cold cache, and it needs network or module-cache access.
- Environment variables like `GOFLAGS` and `GOWORK` change the results. The test has to neutralise them with `t.Setenv`.
- If `go` is missing, `Package.importer()` falls back to `importer.Default()` only when `p.exports == nil`. In `Lint` the resolvers are always set, so a missing `go` would just make `goList` fail and the type information would be silently lost again.

**Alternatives that avoid calling `go`:**
1. **`go/importer.ForCompiler(fset, "source", nil)`**. This is in the stdlib. It type-checks dependencies from source using `go/build`. It needs no `go` binary or export data, so there is no version-mismatch risk. The drawbacks are that it is slower, and `go/build` handles modules poorly. It does call `go list` internally for module-mode lookups, so it isn't fully independent of the toolchain either.
2. **`golang.org/x/tools/go/packages`**. This is the standard approach and what gopls and golangci-lint use. It also runs `go list` under the hood, so it has the same dependency on the toolchain, but it handles modules, vendoring, build tags and workspaces properly. It isn't strictly stdlib, but it is the official, maintained solution.
3. **A custom `types.Importer` that type-checks from source**, using `go/build` plus `go/parser`. It is a lot of code, and it still needs module resolution (`go.mod`, the module cache, vendoring). That is what `go list` exists to do, so you would be re-implementing it.
4. **`importer.Default()`**, as before. It only works when GOROOT export data is available. Since Go 1.20 the stdlib no longer ships precompiled `.a` files, and `-trimpath` breaks GOROOT lookup. So it only resolved the stdlib, which is the bug in issue #1277.

Pure stdlib can't resolve third-party module imports without either the `go` command or reimplementing module resolution. Any robust fix therefore depends on the toolchain. A reasonable middle ground is to use `x/tools/go/packages`, or to keep this approach and fall back to `importer.ForCompiler(..., "source", nil)` when `go` isn't found. You could also document that a `go` binary is required.

@ccoVeille

Copy link
Copy Markdown
Contributor

Further queries to Copilot suggested something like this


Alternative Approach: Using golang.org/x/tools/go/packages for Import Resolution

Thanks for implementing this fix! I understand the concern about depending on a local go binary and potential version mismatches. Since revive already depends on golang.org/x/tools, here's an alternative approach worth exploring:

The Proposal

Replace the direct go list -export mechanism with golang.org/x/tools/go/packages, which is:

  • The standard solution used by gopls and golangci-lint
  • Already in the dependency tree
  • Module-aware and handles vendoring, build tags, workspaces properly
  • Still calls go under the hood, but abstracts away the complexity and version sensitivity

Implementation Sketch

  1. Create a PackagesImporter wrapper:
type PackagesImporter struct {
    mu   sync.RWMutex
    pkgs map[string]*types.Package
}

func NewPackagesImporter(rootDir string) (*PackagesImporter, error) {
    cfg := &packages.Config{
        Mode: packages.NeedName | packages.NeedTypes | packages.NeedTypesInfo | packages.NeedDeps,
        Dir:  rootDir,
    }
    loaded, err := packages.Load(cfg, "./...")
    if err != nil {
        return nil, err
    }
    
    imp := &PackagesImporter{pkgs: make(map[string]*types.Package)}
    packages.Visit(loaded, nil, func(p *packages.Package) {
        if p.Types != nil {
            imp.pkgs[p.PkgPath] = p.Types
        }
    })
    return imp, nil
}

func (i *PackagesImporter) Import(path string) (*types.Package, error) {
    i.mu.RLock()
    defer i.mu.RUnlock()
    if pkg, ok := i.pkgs[path]; ok {
        return pkg, nil
    }
    return nil, fmt.Errorf("package %q not found", path)
}
  1. Preload packages once per linting session in Linter.lintPackage() or Package.TypeCheck()
  2. Use it as the primary importer, with a fallback chain:
    • PackagesImporter (go/packages-based)
    • importer.ForCompiler(..., "source", nil) (source-based, no version mismatch risk)
    • importer.Default() (current behavior)

Benefits

  • ✅ Avoids tight coupling to a specific go binary version
  • ✅ Handles modules, vendoring, workspaces correctly
  • ✅ Explicit fallback chain means type information is never silently lost
  • ✅ Aligns with standard practice in the Go ecosystem
  • ✅ No new external dependencies

Trade-offs

  • Still invokes go internally (unavoidable for third-party module resolution)
  • Slightly more complex than the current PR approach
  • Requires refactoring to pass package context through TypeCheck()

Feel free to adjust the tone or details!

@ccoVeille

Copy link
Copy Markdown
Contributor

@alexandear what do think about the PR, the concerns I raised, the possible solutions that popped out of AI ?

@danp

danp commented Oct 14, 2026

Copy link
Copy Markdown
Author

I'm guessing from earlier workflow approval on #1894 that it's preferred over this PR, closing this one.

@danp danp closed this Oct 14, 2026
@danp
danp deleted the issue-1277 branch October 14, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Typed rules don't work as expected

2 participants