Issues with executor service instrumentation

[kunalherkal]

We have instrumented our ExecutionContext using Kamon and some of the metrics do not work as expected. Other executor service metrics are working fine but we face issues with these metrics:

1. executor.time-in-queue -> this timer metric is not tracked at all, it is absent.
2. executor.tasks.completed -> value of this counter is always zero.
3. executor.tasks.submitted -> value of this counter is always zero.

I was following this documentation. And we do not override any default config related to executor service instrumentation. Let me know if I am missing something.

Here is how we are instrumenting our ExecutionContexts:

object InstrumentedForkJoinPool {
  def apply(name: String): ExecutionContext = {
    val threadFactory = new ForkJoinWorkerThreadFactory() {
      override def newThread(pool: ForkJoinPool): ForkJoinWorkerThread = {
        val worker = ForkJoinPool.defaultForkJoinWorkerThreadFactory.newThread(pool)
        worker.setName(s"$name-" + worker.getPoolIndex)
        worker
      }
    }
    val pool = new ForkJoinPool(Runtime.getRuntime.availableProcessors, threadFactory, null, true);
    val executorService = ExecutionContext.fromExecutorService(pool)
    ExecutorInstrumentation.instrumentExecutionContext(executorService, name)
  }
}

//usage:
object Application extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = {
    Kamon.init()
    val ec: ExecutionContext = InstrumentedForkJoinPool("custom-threadpool")
    val cs: ContextShift[IO] = IO.contextShift(ec)
    //code that uses ec and cs
  }
}

The Kamon dependencies we use:

"io.kamon" %% "kamon-bundle" % "2.1.8"
"io.kamon" %% "kamon-influxdb" % "2.1.8"
"io.kamon" %% "kamon-prometheus" % "2.1.8"
"io.kamon" % "kanela-agent" % "1.0.7"

Other version details:

scalaVersion: 2.12.8, 
sbtVersion: 1.4.2
openjdk version "11.0.2" 2019-01-15

I also submitted another issue #890 earlier, not sure if these are related.