Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(experiment): Compare bincode against other serialisation formats #7513

Draft
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

aakoshh
Copy link
Contributor

@aakoshh aakoshh commented Feb 25, 2025

Description

Problem*

Related to #7511

Summary*

Comparing the impact on bytecode size of switching to a different serialisation format.

Additional Context

I'm interested in figuring out whether we can use one of the schema-less formats without paying too high a price for a backwards compatible format in terms of bloated bytecode size.

Compiling contracts

Run these commands to compile noir protocol circuits and contracts in aztec-packages after rebuilding nargo:

cargo build -p nargo_cli --release
./target/release/nargo --program-dir ../aztec-packages/noir-projects/noir-protocol-circuits compile --force --silence-warnings --skip-underconstrained-check
./target/release/nargo --program-dir ../aztec-packages/noir-projects/noir-contracts compile --force --silence-warnings --skip-underconstrained-check

Baseline

Recorded the bytecode size with bincode before switching to other implementations:

./scripts/bytecode-sizes/print-bytecode-size.sh ../aztec-packages > ./scripts/bytecode-sizes/bincode.jsonl

Comparing bytecode

Run these commands to record a new measurement (e.g. flexbuffers here) and compare against bincode.

./scripts/bytecode-sizes/print-bytecode-size.sh ../aztec-packages > ./scripts/bytecode-sizes/flexbuffers.jsonl
./scripts/bytecode-sizes/compare-bytecode-size.sh ./scripts/bytecode-sizes/bincode.jsonl ./scripts/bytecode-sizes/flexbuffers.jsonl > ./scripts/bytecode-sizes/bincode-vs-flexbuffers.jsonl
./scripts/bytecode-sizes/plot-bytecode-size.sh ./scripts/bytecode-sizes/bincode-vs-flexbuffers.jsonl

Documentation*

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Compilation Time'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite Current: 67e98a9 Previous: 6f78848 Ratio
global_var_regression_entry_points 2.03 s 0.541 s 3.75

This comment was automatically generated by workflow using github-action-benchmark.

CC: @TomAFrench

@aakoshh aakoshh changed the title test: Compare bincode against other serialisation formats feat(experiment): Compare bincode against other serialisation formats Feb 25, 2025
@aakoshh
Copy link
Contributor Author

aakoshh commented Feb 25, 2025

CBOR shows a stable 5-10% overhead for circuits over 10KB in size; below that the overhead goes up to +170% percent.
bincode-vs-cbor

@aakoshh
Copy link
Contributor Author

aakoshh commented Feb 25, 2025

FlexBuffers has a higher base overhead and doesn't show any trend of converging to a lower value for larger circuits, going from +200% to +700%:
bincode-vs-flexbuffers

@aakoshh aakoshh force-pushed the 7511-compare-program-ser branch from c14a35f to 52da1a6 Compare February 28, 2025 12:38
@aakoshh
Copy link
Contributor Author

aakoshh commented Feb 28, 2025

bincode-vs-protobuf

This is the results of comparing bincode to protobuf: Protobuf is 10%-20% worse until 1KB, then 10%-20% better. Beats CBOR, as expected.

@aakoshh aakoshh force-pushed the 7511-compare-program-ser branch from 52da1a6 to e21fccc Compare February 28, 2025 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant