CNDB-8187 Add metrics about time spent writing SAI (compaction and flush)

[eolivelli]

• Add new table and keyspace metrics to track the time spent in SSTables writing (both compaction and flush).
• Add a new table metric with the time spent on writing SAI

New Table and Keyspace metrics:

Histogram flushTime
Histogram compactionTime
Histogram storageAttachedIndexRebuildTime

Histogram  storageAttachedIndexWritingTimeForIndexBuild
Histogram  storageAttachedIndexWritingTimeForCompaction
Histogram  storageAttachedIndexWritingTimeForFlush
Histogram  storageAttachedIndexWritingTimeForOther

storageAttachedIndexWritingTimeForXX is the time spent in "StorageAttachedIndexWriter#addRow" for operation type XXX

For MemTable flush:

• flushTime is the total time
• storageAttachedIndexWritingTimeForFlush is always smaller than flushTime

For Compaction:

• compactionTime is the total time
• storageAttachedIndexWritingTimeForCompaction is always smaller than compactionTime

For IndexBuild:

• storageAttachedIndexRebuildTime is the total time
• storageAttachedIndexWritingTimeForIndexBuild is always smaller than storageAttachedIndexRebuildTime

storageAttachedIndexWritingTimeForOther is for anything else (Garbage Collection, Scrub....)

[eolivelli]

There is still one problem on Compactors: we use StorageAttachedIndexWriter.addRow() even when rebuilding the indexes, we have to disambiguate this case from the other two cases (memtable flush and compaction activities) otherwise it is not possible to correlate correctly storageAttachedIndexWritingTime with the activities that are going on in the node.

[sonarcloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
95.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[jasonstack]

CNDB has CompactionTaskMetrics#duration

[eolivelli]

that's another thing, I initially thought I had to go there, but these metrics are more low level

CompactionTaskMetrics#duration is more about the overall "task", including etcd and other stuff.
I want this metric because it is comparable with the other storageAttachedIndexWritingTimeForCompaction

https://github.com/riptano/cndb/blob/46688e90e2444bde0261a3d499ab6f237a6a306d/cndb-tasks/src/main/java/com/datastax/cndb/compactor/CompactionTaskMetrics.java#L206