[ci] pre-calculating total amulet balance #2407

ray-roestenburg-da · 2025-09-24T19:29:48Z

Calculating total amulet balance for rounds during aggregation to improve performance of queries in v0/total-amulet-balance.

Note: Only implemented new rounds onward, which I think is enough. (otherwise will have to run a catch up process to add old rounds from round_party_totals table, if we really want total amulet balances by round for all historical rounds).
Responding with NotFound for total amulet balances that have not been calculated.

cluster test

Pull Request Checklist

Cluster Testing

If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

Include any change that might be observable by our partners or affect their deployment in the release notes.
Specify fixed issues with Fixes #n, and mention issues worked on using #n
Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

ray-roestenburg-da · 2025-09-27T11:22:19Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanStore.scala

+              select greatest(
+                   0,
+                   sum_cumulative_change_to_initial_amount_as_of_round_zero -
+                   sum_cumulative_change_to_holding_fees_rate * ($asOfEndOfRound + 1)


Pre-computed sum, so sum_change * ($asOfEndOfRound + 1) instead of sum(change_for_round * ($asOfEndOfRound + 1)): (a+b)*c = ac + bc

ray-roestenburg-da · 2025-09-29T09:45:08Z

/cluster_test

github-actions · 2025-09-29T09:45:40Z

Deploy cluster test triggered for Commit 00c7d69b618b5d33b1b943c953b08c5150ea9cd9 in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/33947

ray-roestenburg-da · 2025-09-29T13:30:23Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/admin/http/HttpScanHandler.scala

@@ -413,8 +413,15 @@ class HttpScanHandler(
            HttpErrorHandler.onGrpcNotFound(s"Data for round ${asOfEndOfRound} not yet computed")
          )
      } yield {
-        definitions.GetTotalAmuletBalanceResponse(
-          Codec.encode(total)
+        total.fold(


Changed to respond with NotFound for rounds that were not computed. The change in this PR calculates these going forward. Rather 'error-out' with NotFound than fallback to a very slow query. If we get a signal on this from logs that there is a lot of requests for older rounds, we can put some effort in computing all rounds in the round_total_amulet_balance table, but hopefully that will not be needed.

(the UI already uses getInstrument and this only uses getTotalAmuletBalance if holdingfees are deducted.)

to improve performance of v0/wallet-balance, v0/total-amulet-balance, /v0/top-providers-by-app-rewards Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

…work wel for large number of parties. Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

In the case that the efficient total amulet balance has not been computed for an older round. Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da · 2025-09-29T18:43:17Z

/cluster_test

github-actions · 2025-09-29T18:43:44Z

Deploy cluster test triggered for Commit f8748be6b86def950b4c79feaa41f9b0ff6a4482 in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/34047

OriolMunoz-da · 2025-09-30T10:28:09Z

Before I review:

What was the conclusion about dropping the round-based endpoints like this one?
The FE no longer uses the endpoint fixed here. Do we know if something else is using it?

ray-roestenburg-da · 2025-09-30T11:23:37Z

@OriolMunoz-da

We can't drop anything yet.
Logs show that some are still using it (requests are shown to request older rounds within seconds of the last round, which is not something the FE does)

OriolMunoz-da · 2025-09-30T11:43:00Z

@meiersi-da
Does it make sense to include the flag mentioned here? Without it, this endpoint will keep subtracting holding fees (test example)

meiersi-da · 2025-09-30T11:55:22Z

@meiersi-da Does it make sense to include the flag mentioned here? Without it, this endpoint will keep subtracting holding fees (test example)

@OriolMunoz-da : aren't these the "as-of-round zero" endpoints? For these not deducting holding fees is not easily possible, and we said we'll leave them with their current behavior as legacy features.

OriolMunoz-da

I think I'm mostly coming-back-from-vacation-confused 😄

OriolMunoz-da · 2025-09-30T12:19:18Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanStore.scala

+            sql"""
+              select greatest(
+                0,
+                coalesce(


what are you coalescing here?
both sum_cumulative* columns are not null, and you're not left-joining

ah thanks, cumulative_* in round_party_totals are null, probably misremembered. (it can't really hurt but will remove)

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanStore.scala

OriolMunoz-da · 2025-09-30T12:49:06Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/ScanAggregator.scala

+      on conflict (store_id, closed_round)
+      do update set sum_cumulative_change_to_initial_amount_as_of_round_zero = excluded.sum_cumulative_change_to_initial_amount_as_of_round_zero,
+        sum_cumulative_change_to_holding_fees_rate = excluded.sum_cumulative_change_to_holding_fees_rate;


I don't understand when a conflict would happen and why. Why is it safe to replace the amounts?

Unlikely a conflict occurs, but all of this has to be idempotent because of 'reasons outside of my preference ;-)', so just safer, don't want to fail. it's on conflict pk so it would have to mean a previous aggregation already completed.
Since the aggregation calculates this from round_party_totals and most recent active_parties that is created by this aggregation (and all of this is transactional), it's safe to replace the amounts (it would be the same result).

it would be the same result

then you could as well ignore it, right? I was mostly surprised because some lines above it's ignored. No strong feelings one way or the other

ah, right, yes could just do on conflict do nothing, let me do that instead for clarity

OriolMunoz-da · 2025-09-30T12:53:28Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/ScanAggregator.scala

+      create temp table active_parties_for_aggr_rounds on commit drop as
+      select party, aggr_round as aggr_round, active_round from active_parties_before
+      union
+      -- all the parties that were active in the newly aggregated rounds
+      select party, round as aggr_round, round as active_round from temp_cumulative_totals
+      union
+      -- adding the lastClosedRound as aggr_round, for parties that were not active but have been aggregated in that round
+      select party,
+             $lastClosedRound as aggr_round,
+             closed_round as active_round
+      from   active_parties
+      where  store_id = $roundTotalsStoreId;


open and close parentheses on commit drop as (...), this got very confusing when reviewing

Ok I'll add those

OriolMunoz-da · 2025-09-30T12:55:18Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/ScanAggregator.scala

      sqlu"""
+      create temp table active_parties_before on commit drop as
+      select party,
+             (select max(closed_round) from active_parties where store_id = $roundTotalsStoreId) as aggr_round,


this doesn't depend on the rest of the query, have you checked that the query plan doesn't run this for every party?
it might still be better to extract it

why for every party? This happens per aggregation, which is not per party. It aggregates everything for all rounds it finds.

oh sorry you mean the sub select.. let me check to be safe

Postgres is clever enough luckily (1 loop for aggregate and 1 loop for all the parties). the seq scan is fine cause all parties are needed. (one row per party in this table (per store_id))

Another alternative is:

with max_closed as (select max(closed_round) as aggr_round from scan_sv_1.active_parties where store_id = 1) select party, max_closed.aggr_round, closed_round as active_round from scan_sv_1.active_parties, max_closed where store_id = 1

but this has the same explain plan, rather keep it simple with scalar sub-query.

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da · 2025-10-01T10:36:02Z

Have another look please @OriolMunoz-da

OriolMunoz-da

thanks! let me know if the sub-select does something crazy, otherwise lgtm

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch 7 times, most recently from 5666c77 to 7d841c9 Compare September 25, 2025 18:22

ray-roestenburg-da changed the title ~~[ci] pre-calculating balances and leaderboards in aggregation~~ [ci] pre-calculating total amulet balance Sep 26, 2025

ray-roestenburg-da marked this pull request as ready for review September 27, 2025 10:51

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch from 6c5c43d to 05b9c45 Compare September 27, 2025 10:57

ray-roestenburg-da commented Sep 27, 2025

View reviewed changes

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch from 05b9c45 to 00c7d69 Compare September 29, 2025 09:44

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch from 00c7d69 to a709535 Compare September 29, 2025 13:12

ray-roestenburg-da commented Sep 29, 2025

View reviewed changes

ray-roestenburg-da requested review from OriolMunoz-da, moritzkiefer-da and rautenrieth-da September 29, 2025 13:32

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch 4 times, most recently from d1571cf to c076088 Compare September 29, 2025 18:36

ray-roestenburg-da added 7 commits September 29, 2025 18:39

[ci] pre-calculating balances and leaderboards in aggregation

4868340

to improve performance of v0/wallet-balance, v0/total-amulet-balance, /v0/top-providers-by-app-rewards Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Fix for total amulet balance per round.

eb390aa

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Fix for wallet balances per aggregated round.

7292cb2

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Fix calculate balances when parties are not active in rounds.

e9dc6f3

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Fix for provider app rewards leaderboard.

ba20e72

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Removed leaderboard and wallet balance table since it would not …

4f86494

…work wel for large number of parties. Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[static] some fixes in comments.

013d7f0

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da added 3 commits September 29, 2025 18:39

[ci] Minor fix for empty table.

d636236

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Changed getTotalAmuletBalance to respond with NotFound.

fe56f0a

In the case that the efficient total amulet balance has not been computed for an older round. Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci]Fix in test for getTotalAmuletBalance returning Future[Option[T]]

f8748be

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da force-pushed the ray/leader_boards/1322 branch from c076088 to f8748be Compare September 29, 2025 18:39

OriolMunoz-da reviewed Sep 30, 2025

View reviewed changes

ray-roestenburg-da added 4 commits October 1, 2025 10:13

[ci] Add parenthesis for create temp tables.

e1a6d39

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Remove unnecessary coalesce.

6edf3e7

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Add coalesce to aggregation to be safe

59acab9

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

[ci] Better comment

5970486

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da requested a review from OriolMunoz-da October 1, 2025 10:26

OriolMunoz-da approved these changes Oct 1, 2025

View reviewed changes

[ci] review feedback on conflict do nothing

34e7b91

Signed-off-by: Raymond Roestenburg <raymond.roestenburg@digitalasset.com>

ray-roestenburg-da enabled auto-merge (squash) October 1, 2025 11:28

ray-roestenburg-da merged commit d816f69 into main Oct 1, 2025
57 checks passed

ray-roestenburg-da deleted the ray/leader_boards/1322 branch October 1, 2025 11:34

[ci] pre-calculating total amulet balance #2407

[ci] pre-calculating total amulet balance #2407

Uh oh!

Conversation

ray-roestenburg-da commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Cluster Testing

PR Guidelines

Merge Guidelines

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da commented Sep 29, 2025

Uh oh!

github-actions bot commented Sep 29, 2025

Uh oh!

ray-roestenburg-da Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da commented Sep 29, 2025

Uh oh!

github-actions bot commented Sep 29, 2025

Uh oh!

OriolMunoz-da commented Sep 30, 2025

Uh oh!

ray-roestenburg-da commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

OriolMunoz-da commented Sep 30, 2025

Uh oh!

meiersi-da commented Sep 30, 2025

Uh oh!

OriolMunoz-da left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ray-roestenburg-da commented Oct 1, 2025

Uh oh!

OriolMunoz-da left a comment

Choose a reason for hiding this comment

ray-roestenburg-da commented Sep 24, 2025 •

edited

Loading

ray-roestenburg-da Sep 29, 2025 •

edited

Loading

ray-roestenburg-da commented Sep 30, 2025 •

edited

Loading

ray-roestenburg-da Oct 1, 2025 •

edited

Loading

ray-roestenburg-da Oct 1, 2025 •

edited

Loading

ray-roestenburg-da Oct 1, 2025 •

edited

Loading