Description
Description
Hello.
If we call newrelic.New several times we end up using too much memory. Even when we do not keep all those structs in memory at the same time.
That is because everytime we create a new NewRelic struct a new Config struct is created.
That config is then sent to accounts.New here.
In accounts.New
we end up calling config.GetLogger.
In config.GetLogger
we create a new logger if this config has not already created one. This happens only inside newrelic.New
function, not when we create a different NewRelic
struct.
When we create a new StructuredLogger
we end up calling SetDefaultFields. That function calls logrus's log.AddHook.
At last, here is the problem: logrus log is a global/package variable and we end up adding several hooks to it.
Go Version
go version go1.15.6 linux/amd64
Current behavior
Creating and destroying several NewRelic structs with newrelic.New
function uses too much memory.
Expected behavior
I would expect to be able to create and destroy several NewRelic structs with newrelic.New
without using too much memory.
Steps To Reproduce
Steps to reproduce the behavior:
- Create a benchmark calling
newrelic.New
function
// +build unit
package newrelic
var nr *NewRelic
var err error
func BenchmarkNew(b *testing.B) {
cfgOpt := ConfigPersonalAPIKey(testAPIkey)
for n := 0; n < b.N; n++ {
nr, err = New(cfgOpt)
}
}
- Run it with:
go test -tags unit -memprofile=mem.out -v ./newrelic/... -bench=BenchmarkNew -benchmem -benchtime=20s
- Watch your memory usage increase
- Check results with:
go tool pprof -http :8081 mem.out
Additional Context
I'm aware that we should reuse NewRelic struct. I found this issue by accident.
Should we let StructuredLogger.SetDefaultFields
aware of this global behavior of logrus log with a sync.Once variable?
I was able to decrease memory usage with one, but I do not know if this is the right solution.
Thank you