Support for ESM

[tylersmalley]

Since v12, Node.js has supported ESM. We should investigate the best way to support ESM in the Kibana project.

Our node Babel preset current transpiles down to commonjs. Can we instead use ESM globally?
Does APM support ESM? @watson mentioned there might still be an issue hooking into the loader.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[watson]

After just having a chat with @trentm, I think it should be possible to "hack" our implementation of the APM agent to work despite running as ESM (which officially isn't supported by the agent):

1. Since the agent currently doesn't have a way to hook into the ESM loader, we would have to manually call require() for the modules we want to instrument (and that needs to be instrumented in order for context propagation to not break) before they are imported elsewhere in the source code. I think this is doable.
2. Those modules that we want to instrument must not have separate CJS and ESM code. Else we're instrumenting the wrong code.

I think (besides core modules) that we only care about @elastic/elasticsearch and hapi - possibly also bluebird.

[trentm]

Possibly also the following. (I'm reading through Kibana's current "dependencies" set and comparing to modules for which our APM has some sort of instrumentation.)

• elasticsearch
• handlerbars

[watson]

Getting support for ESM in Kibana is becoming exceedingly important as more and more modules that we rely upon from npm has switched to being ESM only. So we're essentially stuck on older non-maintained versions of many of our dependencies which might contain bugs and security vulnerabilities.

@elastic/kibana-operations Is APM the only thing holding us back, or are there other issues?

@trentm any updates on the APM front, or is our best best still to "hack" the implementation as you suggested previously?