Skip to content

Using more cores makes golangci-lint run slower (?) #5149

Open
@njern

Description

@njern

Welcome

  • Yes, I'm using a binary release within 2 latest releases. Only such installations are supported.
  • Yes, I've searched similar issues on GitHub and didn't find any.
  • Yes, I've read the typecheck section of the FAQ.
  • Yes, I've tried with the standalone linter if available (e.g., gocritic, go vet, etc.).
  • I agree to follow this project's Code of Conduct

Description of the problem

I noticed that both when running a "clean" and a "cached" golangci-lint, setting the core limit lower (with the -j option) seems to improve performance, to a point. It is especially noticeable for cached linting runs.

I first noticed this in our own internal repository but I have reproduced it using the golangci-lint repo.

All these commands were run on a Macbook Pro M4 Max (where golangci-lint defaults to using 16 cores).

cache time
default no cache 32.555855417s
-j 4 no cache 29.854271375s
default cache 2.155626167s
-j 4 cache 1.69703725s
golangci-lint run -v (after golangci-lint cache clean)
INFO golangci-lint has version 1.62.0 built with go1.23.3 from 22b58c9 on 2024-11-10T13:53:08Z
INFO [config_reader] Config search paths: [./ /Users/njern/github/golangci-lint /Users/njern/github /Users/njern /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 36 linters: [bodyclose copyloopvar depguard dogsled dupl errcheck errorlint funlen gocheckcompilerdirectives gochecknoinits goconst gocritic gocyclo godox gofmt goimports goprintffuncname gosec gosimple govet ineffassign intrange lll misspell mnd nakedret noctx nolintlint revive staticcheck stylecheck testifylint unconvert unparam unused whitespace]
INFO [loader] Go packages loading at mode 8767 (types_sizes|compiled_files|exports_file|name|deps|files|imports) took 1.96276675s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 44.92375ms
INFO [linters_context/goanalysis] analyzers took 9m25.254223478s with top 10 stages: gocritic: 6m25.524121759s, buildir: 53.260698388s, goimports: 15.812139791s, dupl: 9.018300673s, the_only_name: 7.497279206s, unconvert: 6.270554796s, ctrlflow: 3.958757812s, fact_deprecated: 3.670158088s, printf: 3.645607995s, gosec: 2.984542789s
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/robustio by pattern internal/go
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/mmap by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/testenv by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir test/testdata_etc/abspath by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 1 issues from dir test/testdata_etc/unused_exported/lib by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 112 issues from dir internal/go/cache by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/quoted by pattern internal/go
INFO [runner] Issues before processing: 510, after processing: 0
INFO [runner] Processors filtering stat (in/out): filename_unadjuster: 510/510, invalid_issue: 510/510, skip_dirs: 510/384, exclude: 384/384, identifier_marker: 384/384, skip_files: 510/510, cgo: 510/510, path_prettifier: 510/510, autogenerated_exclude: 384/384, exclude-rules: 384/68, nolint: 68/0
INFO [runner] processing took 18.164126ms with stages: exclude-rules: 6.558334ms, autogenerated_exclude: 3.532374ms, nolint: 2.844458ms, identifier_marker: 2.691625ms, path_prettifier: 1.879708ms, skip_dirs: 602.499µs, invalid_issue: 23.958µs, cgo: 18.667µs, filename_unadjuster: 10.499µs, max_same_issues: 542ns, fixer: 292ns, skip_files: 209ns, uniq_by_line: 209ns, exclude: 209ns, sort_results: 167ns, max_from_linter: 125ns, max_per_file_from_linter: 84ns, source_code: 83ns, path_shortener: 42ns, diff: 42ns, path_prefixer: 0s, severity-rules: 0s
INFO [runner] linters took 30.540269417s with stages: goanalysis_metalinter: 30.522071875s
INFO File cache stats: 426 entries of total size 917.5KiB
INFO Memory: 325 samples, avg is 2235.9MB, max is 3410.6MB
INFO Execution took 32.555855417s
golangci-lint run -v -j 4 (after golangci-lint cache clean)
INFO golangci-lint has version 1.62.0 built with go1.23.3 from 22b58c9 on 2024-11-10T13:53:08Z
INFO [config_reader] Config search paths: [./ /Users/njern/github/golangci-lint /Users/njern/github /Users/njern /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 36 linters: [bodyclose copyloopvar depguard dogsled dupl errcheck errorlint funlen gocheckcompilerdirectives gochecknoinits goconst gocritic gocyclo godox gofmt goimports goprintffuncname gosec gosimple govet ineffassign intrange lll misspell mnd nakedret noctx nolintlint revive staticcheck stylecheck testifylint unconvert unparam unused whitespace]
INFO [loader] Go packages loading at mode 8767 (files|imports|name|compiled_files|deps|exports_file|types_sizes) took 1.411180584s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 9.101208ms
INFO [linters_context/goanalysis] analyzers took 2m47.626802207s with top 10 stages: gocritic: 1m28.24865404s, buildir: 18.871015413s, goimports: 11.128155453s, dupl: 7.046515498s, the_only_name: 6.109316217s, unconvert: 5.979380999s, fact_deprecated: 1.10383392s, printf: 1.040706755s, inspect: 946.219155ms, gofmt: 928.399373ms
INFO [runner/skip_dirs] Skipped 112 issues from dir internal/go/cache by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/quoted by pattern internal/go
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/robustio by pattern internal/go
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/mmap by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/testenv by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir test/testdata_etc/abspath by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 1 issues from dir test/testdata_etc/unused_exported/lib by pattern test/testdata_etc
INFO [runner] Issues before processing: 510, after processing: 0
INFO [runner] Processors filtering stat (in/out): filename_unadjuster: 510/510, path_prettifier: 510/510, exclude-rules: 384/68, nolint: 68/0, autogenerated_exclude: 384/384, identifier_marker: 384/384, exclude: 384/384, cgo: 510/510, invalid_issue: 510/510, skip_files: 510/510, skip_dirs: 510/384
INFO [runner] processing took 16.580957ms with stages: exclude-rules: 6.523333ms, nolint: 2.763708ms, identifier_marker: 2.504375ms, autogenerated_exclude: 2.314209ms, path_prettifier: 1.828168ms, skip_dirs: 592.249µs, invalid_issue: 24.751µs, cgo: 19.208µs, filename_unadjuster: 9.583µs, max_same_issues: 291ns, fixer: 208ns, sort_results: 208ns, skip_files: 125ns, uniq_by_line: 125ns, exclude: 84ns, diff: 83ns, source_code: 83ns, max_from_linter: 82ns, severity-rules: 42ns, path_shortener: 42ns, max_per_file_from_linter: 0s, path_prefixer: 0s
INFO [runner] linters took 28.425245709s with stages: goanalysis_metalinter: 28.408623375s
INFO File cache stats: 426 entries of total size 917.5KiB
INFO Memory: 295 samples, avg is 1534.9MB, max is 2623.2MB
INFO Execution took 29.854271375s
golangci-lint run -v (with cache)

Ran this five times, execution time ranged between 2,14s and 2,37s.

INFO golangci-lint has version 1.62.0 built with go1.23.3 from 22b58c9 on 2024-11-10T13:53:08Z
INFO [config_reader] Config search paths: [./ /Users/njern/github/golangci-lint /Users/njern/github /Users/njern /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 36 linters: [bodyclose copyloopvar depguard dogsled dupl errcheck errorlint funlen gocheckcompilerdirectives gochecknoinits goconst gocritic gocyclo godox gofmt goimports goprintffuncname gosec gosimple govet ineffassign intrange lll misspell mnd nakedret noctx nolintlint revive staticcheck stylecheck testifylint unconvert unparam unused whitespace]
INFO [loader] Go packages loading at mode 8767 (files|name|types_sizes|compiled_files|exports_file|deps|imports) took 1.206394916s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 41.323542ms
INFO [linters_context/goanalysis] analyzers took 0s with no stages
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/quoted by pattern internal/go
INFO [runner/skip_dirs] Skipped 1 issues from dir test/testdata_etc/unused_exported/lib by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/robustio by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir test/testdata_etc/abspath by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/mmap by pattern internal/go
INFO [runner/skip_dirs] Skipped 112 issues from dir internal/go/cache by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/testenv by pattern internal/go
INFO [runner] Issues before processing: 510, after processing: 0
INFO [runner] Processors filtering stat (in/out): invalid_issue: 510/510, autogenerated_exclude: 384/384, nolint: 68/0, path_prettifier: 510/510, skip_files: 510/510, exclude: 384/384, cgo: 510/510, exclude-rules: 384/68, filename_unadjuster: 510/510, skip_dirs: 510/384, identifier_marker: 384/384
INFO [runner] processing took 16.535252ms with stages: exclude-rules: 6.445667ms, nolint: 2.803958ms, identifier_marker: 2.499625ms, autogenerated_exclude: 2.429708ms, path_prettifier: 1.753499ms, skip_dirs: 551.959µs, invalid_issue: 24.667µs, cgo: 18.334µs, filename_unadjuster: 5.459µs, max_same_issues: 750ns, fixer: 292ns, sort_results: 209ns, diff: 209ns, skip_files: 208ns, max_from_linter: 167ns, exclude: 166ns, uniq_by_line: 125ns, source_code: 125ns, path_shortener: 83ns, severity-rules: 42ns, max_per_file_from_linter: 0s, path_prefixer: 0s
INFO [runner] linters took 899.719083ms with stages: goanalysis_metalinter: 883.152667ms
INFO File cache stats: 0 entries of total size 0B
INFO Memory: 23 samples, avg is 54.1MB, max is 93.8MB
INFO Execution took 2.155626167s
golangci-lint run -v -j 4 (with cache)

Also ran this five times, execution time ranged from 1,43s to 1,69s

INFO golangci-lint has version 1.62.0 built with go1.23.3 from 22b58c9 on 2024-11-10T13:53:08Z
INFO [config_reader] Config search paths: [./ /Users/njern/github/golangci-lint /Users/njern/github /Users/njern /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 36 linters: [bodyclose copyloopvar depguard dogsled dupl errcheck errorlint funlen gocheckcompilerdirectives gochecknoinits goconst gocritic gocyclo godox gofmt goimports goprintffuncname gosec gosimple govet ineffassign intrange lll misspell mnd nakedret noctx nolintlint revive staticcheck stylecheck testifylint unconvert unparam unused whitespace]
INFO [loader] Go packages loading at mode 8767 (compiled_files|deps|files|types_sizes|exports_file|imports|name) took 1.38926575s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 9.9845ms
INFO [linters_context/goanalysis] analyzers took 0s with no stages
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/quoted by pattern internal/go
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/mmap by pattern internal/go
INFO [runner/skip_dirs] Skipped 112 issues from dir internal/go/cache by pattern internal/go
INFO [runner/skip_dirs] Skipped 1 issues from dir test/testdata_etc/unused_exported/lib by pattern test/testdata_etc
INFO [runner/skip_dirs] Skipped 3 issues from dir internal/go/testenv by pattern internal/go
INFO [runner/skip_dirs] Skipped 2 issues from dir internal/go/robustio by pattern internal/go
INFO [runner/skip_dirs] Skipped 3 issues from dir test/testdata_etc/abspath by pattern test/testdata_etc
INFO [runner] Issues before processing: 510, after processing: 0
INFO [runner] Processors filtering stat (in/out): identifier_marker: 384/384, exclude-rules: 384/68, invalid_issue: 510/510, path_prettifier: 510/510, skip_dirs: 510/384, autogenerated_exclude: 384/384, cgo: 510/510, exclude: 384/384, filename_unadjuster: 510/510, skip_files: 510/510, nolint: 68/0
INFO [runner] processing took 16.082209ms with stages: exclude-rules: 6.301083ms, identifier_marker: 2.826209ms, nolint: 2.734626ms, autogenerated_exclude: 1.970082ms, path_prettifier: 1.677959ms, skip_dirs: 533.166µs, cgo: 17.167µs, invalid_issue: 14.917µs, filename_unadjuster: 4.667µs, max_same_issues: 792ns, diff: 249ns, sort_results: 209ns, fixer: 209ns, skip_files: 207ns, max_from_linter: 166ns, uniq_by_line: 125ns, source_code: 84ns, severity-rules: 83ns, path_shortener: 83ns, path_prefixer: 42ns, max_per_file_from_linter: 42ns, exclude: 42ns
INFO [runner] linters took 289.921334ms with stages: goanalysis_metalinter: 273.804875ms
INFO File cache stats: 0 entries of total size 0B
INFO Memory: 18 samples, avg is 38.0MB, max is 100.3MB
INFO Execution took 1.69703725s

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.62.0 built with go1.23.3 from 22b58c9 on 2024-11-10T13:53:08Z

Configuration

# paste configuration file or CLI flags here

Go environment

$ go version && go env
go version go1.23.3 darwin/arm64
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/njern/Library/Caches/go-build'
GOENV='/Users/njern/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/njern/go/pkg/mod'
GOOS='darwin'
GOPATH='/Users/njern/go'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.23.3/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.23.3/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.3'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/njern/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/njern/github/golangci-lint/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/23/rl5hsyxx4klg_9_7l2ymybp00000gn/T/go-build4225367625=/tmp/go-build -gno-record-gcc-switches -fno-common'

Validation

  • Yes, I've included all information above (version, config, etc.).

Supporter

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions