powdr: end-to-end prover #7

gballet · 2025-03-13T17:22:59Z

This creates glue between powdr and zeam, so that the block data can be passed as private inputs. The publics extraction and verification require more work on the powdr side, so I suggest we merge this PR first.

TODO before merging:

ensure that prove also rebuilds the guest programs
cleanup comments and debug text
correctly handle panics when an error is returned

Copilot

Pull Request Overview

This PR creates a new library to establish bindings between powdr and zeam, allowing block data to be passed as private inputs.

Added a Cargo.toml for the new host library.
Introduced an FFI entry point (prove) in the host library that wraps session setup and execution.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
pkgs/state-transition-runtime/src/powdr/host/Cargo.toml	New package configuration for the zeam-prover-host-powdr library
pkgs/state-transition-runtime/src/powdr/host/src/lib.rs	Implementation of the FFI prove function linking powdr Sessions

Copilot · 2025-03-13T17:23:50Z

pkgs/state-transition-runtime/src/powdr/host/src/lib.rs

+        if !serialized.is_null() {
+            std::slice::from_raw_parts(output, len)


The _output_slice initialization is using the 'serialized' pointer instead of 'output'. Update the condition to check if 'output' is null, and use 'output_len' in place of 'len' to correctly reflect the intended slice length.

Suggested change

if !serialized.is_null() {

std::slice::from_raw_parts(output, len)

if !output.is_null() {

std::slice::from_raw_parts(output, output_len)

gballet · 2025-03-30T20:40:04Z

Note that backtraces report both rust and zig code !

thread '<unnamed>' panicked at src/lib.rs:44:9:
path does not exit
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread '<unnamed>' panicked at library/core/src/panicking.rs:218:5:
panic in a function that cannot unwind
stack backtrace:
   0:          0x1642962 - std::backtrace_rs::backtrace::libunwind::trace::hd55534e481fa4cba
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
   1:          0x1642962 - std::backtrace_rs::backtrace::trace_unsynchronized::he5d0dbcdee39ac25
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
   2:          0x1642962 - std::sys::backtrace::_print_fmt::hd8b10669df281c74
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/sys/backtrace.rs:66:9
   3:          0x1642962 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h2b56cf5291b3667a
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/sys/backtrace.rs:39:26
   4:          0x160f873 - core::fmt::rt::Argument::fmt::h7d04f9725982d86a
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/fmt/rt.rs:185:76
   5:          0x160f873 - core::fmt::write::h750c4757982490e3
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/fmt/mod.rs:1449:21
   6:          0x163ef83 - std::io::Write::write_fmt::h4aaf30de29f6dc2b
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/io/mod.rs:1890:15
   7:          0x16427b2 - std::sys::backtrace::BacktraceLock::print::h1b35dc075e3a8fee
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/sys/backtrace.rs:42:9
   8:          0x1643a12 - std::panicking::default_hook::{{closure}}::hb12e4b8e4fe4a1fd
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/panicking.rs:298:22
   9:          0x1643815 - std::panicking::default_hook::h4204d9ba1ec820cc
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/panicking.rs:325:9
  10:          0x1644432 - std::panicking::rust_panic_with_hook::h64733910b09a62a0
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/panicking.rs:831:13
  11:          0x1644186 - std::panicking::begin_panic_handler::{{closure}}::h9e662ad4e13fe8d5
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/panicking.rs:697:13
  12:          0x1642e59 - std::sys::backtrace::__rust_end_short_backtrace::h18e5771372e27504
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/sys/backtrace.rs:168:18
  13:          0x1643e4d - rust_begin_unwind
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/std/src/panicking.rs:695:5
  14:          0x160d3cd - core::panicking::panic_nounwind_fmt::runtime::h0ecba1e502c8bbcc
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/panicking.rs:117:22
  15:          0x160d3cd - core::panicking::panic_nounwind_fmt::hb9be14d6da3e6da4
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/intrinsics/mod.rs:3240:9
  16:          0x160d462 - core::panicking::panic_nounwind::h411e261a8a67f825
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/panicking.rs:218:5
  17:          0x160d586 - core::panicking::panic_cannot_unwind::he03ec2bd054fb53c
                               at /rustc/43a2e9d2c72db101f5fedac8b3acb78981b06bf2/library/core/src/panicking.rs:323:5
  18:          0x1608d6b - powdr_prove
  19:          0x15d4bdf - manager.execute_transition
                               at /home/gballet/src/zeam/pkgs/state-proving-manager/src/manager.zig:35:16
  20:          0x15d4bdf - main.main
                               at /home/gballet/src/zeam/pkgs/cli/src/main.zig:58:59
  21:          0x15d8ab5 - start.callMain
                               at /snap/zig/13822/lib/std/start.zig:656:37
  22:          0x15d8ab5 - start.callMainWithArgs
                               at /snap/zig/13822/lib/std/start.zig:616:20
  23:          0x15d8ab5 - start.main
                               at /snap/zig/13822/lib/std/start.zig:631:28
  24:     0x72058a22a1ca - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  25:     0x72058a22a28b - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:360:3
  26:          0x15d06f5 - _start
thread caused non-unwinding panic. aborting.

Copilot

Pull Request Overview

This PR implements an end-to-end prover that creates the integration layer between powdr and zeam, allowing block data to be passed as private inputs (required for #3).

Corrects a typo in the zeam documentation.
Introduces a new FFI function (powdr_prove) to initialize and run the powdr session.
Adds the Cargo.toml configuration for building the host as a static library.

Reviewed Changes

Copilot reviewed 5 out of 11 changed files in this pull request and generated 1 comment.

File	Description
resources/zeam.md	Corrected spelling of "latest_block_header"
pkgs/state-transition-runtime/src/powdr/host/src/lib.rs	Added FFI function powdr_prove for session initialization and proving
pkgs/state-transition-runtime/src/powdr/host/Cargo.toml	Added host configuration for building the static library

Files not reviewed (6)

build.zig: Language not supported
pkgs/cli/src/main.zig: Language not supported
pkgs/state-proving-manager/src/manager.zig: Language not supported
pkgs/state-transition/src/transition.zig: Language not supported
pkgs/state-transition/src/utils.zig: Language not supported
pkgs/types/src/lib.zig: Language not supported

Comments suppressed due to low confidence (1)

resources/zeam.md:31

The typo 'lastest' has been corrected to 'latest' to reflect the correct property name. Verify that all references now match the intended naming.

    - only block's parent_root matched against state's `latest_block_header` hash tree root

Copilot · 2025-03-31T06:53:41Z

pkgs/state-transition-runtime/src/powdr/host/src/lib.rs

+        if !serialized.is_null() {
+            std::slice::from_raw_parts(output, len)


The check for the output slice incorrectly verifies 'serialized' rather than 'output', and uses 'len' instead of 'output_len'. Consider updating the condition to 'if !output.is_null()' and using 'output_len' when creating the slice.

Suggested change

if !serialized.is_null() {

std::slice::from_raw_parts(output, len)

if !output.is_null() {

std::slice::from_raw_parts(output, output_len)

Copilot

Pull Request Overview

This PR introduces an end-to-end prover that bridges powdr and zeam by passing block data as private inputs, addressing part of issue #3.

Corrected a typo in the zeam documentation regarding the state header name.
Added a new FFI function (powdr_prove) in Rust for invoking the prover with byte slices sourced from external pointers.
Introduced a dedicated Cargo.toml for the zeam-prover-host-powdr package and updated the README build instructions accordingly.

Reviewed Changes

Copilot reviewed 6 out of 12 changed files in this pull request and generated 1 comment.

File	Description
resources/zeam.md	Fixed typo in the protocol description regarding state header
pkgs/state-transition-runtime/src/powdr/host/src/lib.rs	Introduced the powdr_prove function with pointer handling
pkgs/state-transition-runtime/src/powdr/host/Cargo.toml	Added package configuration and dependency details
README.md	Updated build instructions and added a demo run section

Files not reviewed (6)

build.zig: Language not supported
pkgs/cli/src/main.zig: Language not supported
pkgs/state-proving-manager/src/manager.zig: Language not supported
pkgs/state-transition/src/transition.zig: Language not supported
pkgs/state-transition/src/utils.zig: Language not supported
pkgs/types/src/lib.zig: Language not supported

pkgs/state-transition-runtime/src/powdr/host/src/lib.rs

gballet · 2025-04-01T20:31:15Z

Blocked on powdr-labs/powdr#2622 and powdr-labs/powdr#2620 for merging, but I can keep working on it in the meantime.

gballet · 2025-04-03T09:40:47Z

g11tech · 2025-04-06T10:45:00Z

pkgs/cli/src/main.zig

@@ -28,5 +42,34 @@ pub fn main() !void {

            try clock.run();
        },
+        .prove => |provecmd| {
+            const state = types.BeamState{


i guess i can replace it by genMockChain

feel free to

g11tech · 2025-04-06T10:49:40Z

pkgs/state-transition/src/transition.zig

@@ -64,6 +64,7 @@ pub fn verify_signatures(signedBlock: types.SignedBeamBlock) !void {
    _ = signedBlock;
 }

+// TODO(gballet) check if beam block needs to be a pointer


it doesn't need to be but zig must be memcping it so i guess pointer would be better

let's do the optimization in another PR, right now it works

pkgs/state-proving-manager/src/manager.zig

g11tech · 2025-04-06T10:57:02Z

pkgs/state-proving-manager/src/manager.zig

-}
+extern fn powdr_prove(serialized: [*]const u8, len: usize, output: [*]u8, output_len: usize, binary_path: [*]const u8, binary_path_length: usize, result_path: [*]const u8, result_path_len: usize) void;
+
+pub const zkvm_configs: []const ZKVMContext = &.{


we can probably transition it to a enum union struct

makes sense, but since there's only one right now, let's leave it as is and change it in a future PR when we support proving more than one zkvm

pkgs/state-proving-manager/src/manager.zig

g11tech · 2025-04-06T11:00:35Z

pkgs/state-proving-manager/src/manager.zig

+    try ssz.serialize(types.BeamSTFProverInput, prover_input, &serialized);
+
+    var output: [256]u8 = undefined;
+    powdr_prove(serialized.items.ptr, serialized.items.len, @ptrCast(&output), 256, zkvm_configs[0].program_path.ptr, zkvm_configs[0].program_path.len, zkvm_configs[0].output_dir.ptr, zkvm_configs[0].output_dir.len);
    return types.BeamSTFProof{};


so proof is still somewhere in the files and not pickedup yet?

yeah atm, this comes with the verifier PR, which will probably cause some changes in what is returned so I'll leave it at that for now

g11tech

lgtm

gballet requested a review from Copilot March 13, 2025 17:22

Copilot AI reviewed Mar 13, 2025

View reviewed changes

gballet mentioned this pull request Mar 28, 2025

build: glue builder and build.zig cleanup #11

Merged

gballet force-pushed the create-powdr-rust-bindings branch from 78b1a59 to 5131f55 Compare March 28, 2025 21:42

gballet changed the title ~~powdr: create prover bindings lib~~ powdr: end-to-end prover Mar 30, 2025

gballet force-pushed the create-powdr-rust-bindings branch 4 times, most recently from 616e553 to a187862 Compare March 30, 2025 20:19

gballet requested a review from Copilot March 31, 2025 06:53

Copilot AI reviewed Mar 31, 2025

View reviewed changes

gballet force-pushed the create-powdr-rust-bindings branch 2 times, most recently from de25aac to 082e10c Compare April 1, 2025 20:29

gballet requested a review from Copilot April 1, 2025 20:30

Copilot AI reviewed Apr 1, 2025

View reviewed changes

pkgs/state-transition-runtime/src/powdr/host/src/lib.rs Outdated Show resolved Hide resolved

gballet force-pushed the create-powdr-rust-bindings branch from 082e10c to 0be6f69 Compare April 3, 2025 09:17

gballet force-pushed the create-powdr-rust-bindings branch 3 times, most recently from e4419e2 to ab53efe Compare April 5, 2025 13:05

gballet marked this pull request as ready for review April 5, 2025 19:50

gballet mentioned this pull request Apr 5, 2025

Rebase powdr on latest #15

Closed

gballet force-pushed the create-powdr-rust-bindings branch from ab53efe to 06d7133 Compare April 5, 2025 21:23

g11tech reviewed Apr 6, 2025

View reviewed changes

pkgs/state-proving-manager/src/manager.zig Outdated Show resolved Hide resolved

g11tech reviewed Apr 6, 2025

View reviewed changes

pkgs/state-proving-manager/src/manager.zig Outdated Show resolved Hide resolved

g11tech reviewed Apr 6, 2025

View reviewed changes

pkgs/state-proving-manager/src/manager.zig Outdated Show resolved Hide resolved

g11tech reviewed Apr 6, 2025

View reviewed changes

powdr: create prover bindings lib

40f94ed

gballet force-pushed the create-powdr-rust-bindings branch from 06d7133 to 40f94ed Compare April 6, 2025 20:36

g11tech approved these changes Apr 7, 2025

View reviewed changes

g11tech merged commit ee8140a into main Apr 7, 2025
6 checks passed

g11tech deleted the create-powdr-rust-bindings branch April 12, 2025 16:43

		if !serialized.is_null() {
		std::slice::from_raw_parts(output, len)

powdr: end-to-end prover #7

powdr: end-to-end prover #7

Uh oh!

Conversation

gballet commented Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Mar 13, 2025

Choose a reason for hiding this comment

Uh oh!

gballet commented Mar 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

gballet commented Apr 1, 2025

Uh oh!

gballet commented Apr 3, 2025

Uh oh!

g11tech Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

gballet Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

g11tech Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

gballet Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

g11tech Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

gballet Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

g11tech Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gballet Apr 6, 2025

Choose a reason for hiding this comment

Uh oh!

g11tech left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gballet commented Mar 13, 2025 •

edited

Loading

g11tech Apr 6, 2025 •

edited

Loading