Skip to content

Creating several NewRelic structs uses too much memory #576

Open
@Tarliton

Description

@Tarliton

Description

Hello.
If we call newrelic.New several times we end up using too much memory. Even when we do not keep all those structs in memory at the same time.

That is because everytime we create a new NewRelic struct a new Config struct is created.
That config is then sent to accounts.New here.
In accounts.New we end up calling config.GetLogger.
In config.GetLogger we create a new logger if this config has not already created one. This happens only inside newrelic.New function, not when we create a different NewRelic struct.
When we create a new StructuredLogger we end up calling SetDefaultFields. That function calls logrus's log.AddHook.
At last, here is the problem: logrus log is a global/package variable and we end up adding several hooks to it.

Go Version

go version go1.15.6 linux/amd64

Current behavior

Creating and destroying several NewRelic structs with newrelic.New function uses too much memory.

Expected behavior

I would expect to be able to create and destroy several NewRelic structs with newrelic.New without using too much memory.

Steps To Reproduce

Steps to reproduce the behavior:

  1. Create a benchmark calling newrelic.New function
// +build unit

package newrelic

var nr *NewRelic
var err error

func BenchmarkNew(b *testing.B) {
	cfgOpt := ConfigPersonalAPIKey(testAPIkey)

	for n := 0; n < b.N; n++ {
		nr, err = New(cfgOpt)
	}

}
  1. Run it with:
go test -tags unit -memprofile=mem.out -v ./newrelic/... -bench=BenchmarkNew -benchmem -benchtime=20s
  1. Watch your memory usage increase
  2. Check results with:
go tool pprof -http :8081 mem.out
  1. VIEW ->Top
    1nr

Additional Context

I'm aware that we should reuse NewRelic struct. I found this issue by accident.
Should we let StructuredLogger.SetDefaultFields aware of this global behavior of logrus log with a sync.Once variable?
I was able to decrease memory usage with one, but I do not know if this is the right solution.
2nr

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingclient-gopinnedPin an issue to prevent auto-closing by a bot.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions