Proposal for a benchmark execution goal with JVM implementation

[alonsodomin]

This PR adds a new main goal to Pants under the alias bench (or may be better to put it under experimental-bench) with the sole purpose to execute microbenchmaks.

The PR provides as well an implementation for the JVM based on JMH (the most popular framework) and Scalameter.

The core goal implementation is blatantly inspired on the test goal (removing support for coverage data and debugging), and, if adding a new goal is considered unnecessary, the implementations could switch to be based off test instead. There is no consensus among developers of different communities and tooling on whether benchmarks are just tests or if they are a separate thing all together. For example in Go the standard is to use go test -bench=. while in Rust the command used is cargo bench.

The choice of putting them as separate comes from preventing unnecessary wait times when running tests (and not being aware that benchmarks are among those) and making the end user be intentional about their purpose. Happy to get rid of the goal though if we find that it as bloat.

The rest of the implementation follows a similar pattern as the one used for supporting JUnit: Having support for running JMH benchmarks in the 3 JVM languages supported with an extra for Scalameter.

tailor implementation is not based on a general consensus either, the choice of using Benchmark as a suffix for JMH and Bench for Scalameter is just made for convenience.

[ci skip-build-wheels]
[ci skip-rust]

[tdyas]

What do bazel and other build systems comparable to Pants provide for invoking benchmarks?

[alonsodomin]

I only looked a bit into a few impls in Bazel and it seems the approach the contributors are following are to execute them with the run command.

For example:

• Go rules: The recommendation is using the go_test target and passing the passthrough arg -test.bench=.: https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/rules.md#go_test
• JMH Java rules: Using a specific jmh_java_benchmarks target but still run them using bazel run: https://github.com/buchgr/rules_jmh
• Rust rules: No support to run benchmarks so far, even though out of those is the one that has decided on standardizing a bit and having a specific command for running them (cargo bench).

Haven't found any support at all in Buck. Please does support Go benchmarks via a specific target named go_benchmark and from the docs I gather these are run using the test goal instead: https://please.build/plugins.html#go

So support is very patchy. This is obviously something not standardized across language toolchains and much less on monorepo build tools.