Need to implement a benchmark to catch performance issues like https://github.com/gammapy/gammapy/pull/5979/