Skip to content

Conversation

@qwang98
Copy link
Collaborator

@qwang98 qwang98 commented Dec 11, 2025

PR'ed to ovm-11-16, which is the Powdr dependency branch of our latest reth branch 1.4.1-powdr-11-16.

This PR answers the question "For typical blocks, what is the distribution of the amount of removed columns during apc compilation? This would inform how much we can save in tracegen time." from issue #3482 by collecting the before and after optimization stats of APC subs by instruction. The hope is that we see a large portion of records/APC rows with all columns optimized away, and thus skip those records entirely (because passing records from CPU to GPU takes 80% of tracegen time). To achieve this, this PR:

  1. Collects before and after optimization # of columns for each instruction of each APC.
  2. Aggregate instructions of the same AIR across all APCs, because they have the same number of before optimization columns.
  • There's a caveat that not all APCs are equally important because some are executed more than others, and therefore we also need the frequency of execution for each APC in order to aggregate the same AIR across APCs. This data is not guaranteed from PGO as we don't necessarily use the cell mode. (In fact, Reth uses instruction PGO because the cell one is prohibitively slow).
  • To achieve this, this PR collects execution counts after preflight execution and merge it with the subs information for each APC.
  1. Export these data as a JSON file to Python, which outputs:
  • Per AIR plot aggregated over all APCs, weighted by APC execution count. (The graphs are named as [air_name].png)
  • Summary plot aggregated over all AIRs over all APCs, weighted by AIR width. This is because we'd expect the cost of passing over a record be proportional to the trace width.

This test is run over both Keccak and Reth with APC=30 (which is the "max" we can run with recursion). All JSON and plot outputs are pushed to this PR.

@qwang98 qwang98 mentioned this pull request Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants