feat(settlement): add metrics + Grafana panels for the settlement service - #625
Merged
Conversation
Instrument the settle loop with Prometheus metrics, keyed at the service layer so they are agnostic to the proving backend. The headline metric is proof-generation time; also tracked are state-update time, end-to-end settle time, batch size, settled-block cursor, and settle/failure counters. Proof-related timings (proof_generation_seconds, settle_batch_seconds) carry a `proof_type` label (e.g. sp1, mock) via a new ProvingBackend::proof_type() so they can be compared across backends. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seven panels for the settlement service metrics: proof-generation and settle-batch duration (split by proof_type), state-update latency, blocks-per-batch, settlement throughput, failure rate, and the current settled-block cursor. Duration panels read the summary quantile labels directly, matching the existing RPC-latency panels. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.30.
| Benchmark suite | Current: 32df2a5 | Previous: bbe2576 | Ratio |
|---|---|---|---|
FinalityStatus/compress |
1 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
TypedTransactionExecutionInfo/decompress |
3581 ns/iter (± 64) |
2673 ns/iter (± 149) |
1.34 |
TrieHistoryEntry/decompress |
260 ns/iter (± 11) |
183 ns/iter (± 16) |
1.42 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @kariy
Codec benchmark diff vs
|
| Benchmark | Baseline (ns) | Current (ns) | Δ |
|---|---|---|---|
CompiledClass(fixture)/compress |
2485031 | 2679052 | +7.81% |
CompiledClass(fixture)/decompress |
2624721 | 2803165 | +6.80% |
ExecutionCheckpoint/compress |
27 | 31 | +14.81% |
ExecutionCheckpoint/decompress |
23 | 23 | +0.00% |
PruningCheckpoint/compress |
27 | 31 | +14.81% |
PruningCheckpoint/decompress |
23 | 23 | +0.00% |
VersionedHeader/compress |
623 | 605 | -2.89% |
VersionedHeader/decompress |
744 | 839 | +12.77% |
StoredBlockBodyIndices/compress |
70 | 74 | +5.71% |
StoredBlockBodyIndices/decompress |
36 | 34 | -5.56% |
StorageEntry/compress |
122 | 143 | +17.21% |
StorageEntry/decompress |
118 | 142 | +20.34% |
ContractNonceChange/compress |
123 | 145 | +17.89% |
ContractNonceChange/decompress |
204 | 233 | +14.22% |
ContractClassChange/compress |
167 | 183 | +9.58% |
ContractClassChange/decompress |
219 | 252 | +15.07% |
ContractStorageEntry/compress |
131 | 152 | +16.03% |
ContractStorageEntry/decompress |
281 | 331 | +17.79% |
GenericContractInfo/compress |
116 | 129 | +11.21% |
GenericContractInfo/decompress |
83 | 104 | +25.30% |
Felt/compress |
69 | 80 | +15.94% |
Felt/decompress |
45 | 55 | +22.22% |
BlockHash/compress |
69 | 80 | +15.94% |
BlockHash/decompress |
45 | 58 | +28.89% |
TxHash/compress |
72 | 80 | +11.11% |
TxHash/decompress |
46 | 55 | +19.57% |
ClassHash/compress |
70 | 80 | +14.29% |
ClassHash/decompress |
45 | 55 | +22.22% |
CompiledClassHash/compress |
69 | 80 | +15.94% |
CompiledClassHash/decompress |
45 | 55 | +22.22% |
BlockNumber/compress |
43 | 47 | +9.30% |
BlockNumber/decompress |
23 | 24 | +4.35% |
TxNumber/compress |
42 | 47 | +11.90% |
TxNumber/decompress |
23 | 24 | +4.35% |
FinalityStatus/compress |
0 | 1 | +Infinity% |
FinalityStatus/decompress |
9 | 10 | +11.11% |
TypedTransactionExecutionInfo/compress |
14558 | 16094 | +10.55% |
TypedTransactionExecutionInfo/decompress |
2673 | 3581 | +33.97% |
VersionedContractClass/compress |
366 | 380 | +3.83% |
VersionedContractClass/decompress |
654 | 797 | +21.87% |
MigratedCompiledClassHash/compress |
117 | 143 | +22.22% |
MigratedCompiledClassHash/decompress |
117 | 139 | +18.80% |
ContractInfoChangeList/compress |
1490 | 1360 | -8.72% |
ContractInfoChangeList/decompress |
2050 | 2102 | +2.54% |
BlockChangeList/compress |
645 | 586 | -9.15% |
BlockChangeList/decompress |
844 | 845 | +0.12% |
ReceiptEnvelope/compress |
25381 | 29965 | +18.06% |
ReceiptEnvelope/decompress |
5335 | 6114 | +14.60% |
TrieDatabaseValue/compress |
138 | 165 | +19.57% |
TrieDatabaseValue/decompress |
229 | 234 | +2.18% |
TrieHistoryEntry/compress |
248 | 281 | +13.31% |
TrieHistoryEntry/decompress |
183 | 260 | +42.08% |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #625 +/- ##
==========================================
- Coverage 73.32% 68.11% -5.21%
==========================================
Files 209 333 +124
Lines 23132 46615 +23483
==========================================
+ Hits 16961 31751 +14790
- Misses 6171 14864 +8693 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Prometheus metrics to the embedded settlement service, plus a Settlement row on the Grafana overview dashboard. The headline metric is proof-generation time, with other settlement stages and outcomes tracked alongside.
Metrics (
katana_settlement_*)Instrumented at the service layer so they are agnostic to the proving backend:
proof_generation_secondsprove()— attestation build + proving (the dominant cost)settle_batch_secondsstate_update_secondsupdate_statetx submit + confirmationblocks_per_batchbatches_settled_total/blocks_settled_totalsettlement_failures_totalsettled_blockThe two proof-related timings carry a
proof_typelabel (sp1/mock) via a newProvingBackend::proof_type(), so they can be broken down and compared across backends.Because the global Prometheus recorder is already installed at node startup, these auto-register on the existing metrics endpoint — no additional wiring.
Dashboard
A 7-panel Settlement row in
monitoring/grafana/dashboards/overview.json. Duration panels read the exporter's summaryquantilelabels directly (p50/p90/p99), matching the existing RPC-latency panels. Inserted as pure additions with no reformatting of existing panels.Checks
cargo +nightly-2025-02-20 fmt --all— clean-p katana-settlement --all-targetswith the repo's deny flags) — clean🤖 Generated with Claude Code