Pre develop 0.16.0 by fractasy · Pull Request #789 · 0xPolygonHermez/zisk

fractasy · 2026-02-15T14:20:51Z

No description provided.

Copilot

Pull request overview

This PR prepares the codebase for the 0.16.0 development cycle by introducing new DMA-focused assembly routines, expanding shared-memory + streaming infrastructure (including precompile hints), and updating distributed/coordinator APIs and tooling to support new execution/metadata flows.

Changes:

Add optimized x86_64 DMA assembly helpers (fast memcpy + DMA encode) and wire them into the emulator build.
Introduce shared-memory reader/writer upgrades and new hints sinks/sources (file + shmem + streaming).
Update distributed gRPC/proto + coordinator/worker configs to support inputs/hints modes, streaming, and capacity semantics.

Reviewed changes

Copilot reviewed 161 out of 665 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
emulator-asm/src/dma/*	Adds DMA assembly routines, constants, and a local test Makefile
emulator-asm/Makefile	Links DMA objects into emulator build
emulator-asm/asm-runner/src/*	Adds shmem reader, ring-buffer writer, hints sinks, and naming helpers
distributed/crates/*	Updates proto/types to support hints + streaming and capacity changes
core/src/*	Switches ELF handling to byte-slices, adds new op types/flags, updates SHA256 helper
common/src/*	Refactors IO into stdin/stream, adds stream abstractions, changes trace mem_reads to Cow
cli/src/*	Updates commands for new SDK APIs, inputs/hints URIs, adds SNARK verify/prove commands
book/getting_started/*	Updates docs for new project layout and commands
.vscode/settings.json	Updates editor setting
.github/workflows/pr.yml	Adjusts CI timeouts and SHM size

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-15T14:23:44Z

+    let state_u32: &mut [u32; 8] = unsafe { &mut *(state.as_mut_ptr() as *mut [u32; 8]) };
+    let input_u8: &[GenericArray<u8, U64>; 1] =
+        unsafe { &*(input.as_ptr() as *const [GenericArray<u8, U64>; 1]) };
+    compress256(state_u32, input_u8);


This changes SHA-256 block/state byte ordering compared to the previous implementation (which explicitly constructed big-endian bytes and split u64->u32 in a controlled order). The new as_ptr() reinterprets host-endian memory directly, which will produce different results on little-endian systems unless the upstream state/input layout was also changed accordingly. To keep correctness, either preserve explicit endian conversions or document/guarantee the exact in-memory layout expected by callers and ensure they provide bytes/words in that layout.

Suggested change

let state_u32: &mut [u32; 8] = unsafe { &mut *(state.as_mut_ptr() as *mut [u32; 8]) };

let input_u8: &[GenericArray<u8, U64>; 1] =

unsafe { &*(input.as_ptr() as *const [GenericArray<u8, U64>; 1]) };

compress256(state_u32, input_u8);

// Convert the 4x u64 state into 8x u32 words in a fixed (big-endian) order.

let mut state_u32 = [0u32; 8];

for (i, word) in state.iter().cloned().enumerate() {

// High 32 bits first, then low 32 bits, independent of host endianness.

state_u32[2 * i] = (word >> 32) as u32;

state_u32[2 * i + 1] = word as u32;

}

// Convert the 8x u64 input words into a single 64-byte block in big-endian order.

let mut block = GenericArray::<u8, U64>::default();

for (i, word) in input.iter().enumerate() {

let bytes = word.to_be_bytes();

let offset = i * 8;

block[offset..offset + 8].copy_from_slice(&bytes);

}

// Compress a single 512-bit block.

compress256(&mut state_u32, std::slice::from_ref(&block));

// Convert the updated 8x u32 state back into 4x u64 words using the inverse mapping.

for i in 0..4 {

let hi = state_u32[2 * i] as u64;

let lo = state_u32[2 * i + 1] as u64;

state[i] = (hi << 32) | lo;

}

Copilot · 2026-02-15T14:23:46Z

        "editor.defaultFormatter": "rust-lang.rust-analyzer",
        "editor.formatOnSave": true,
-        "editor.hover.enabled": true
+        "editor.hover.enabled": "on"


editor.hover.enabled is a boolean setting in VS Code. The string value "on" is likely invalid and may be ignored. Change it to true/false (or remove if relying on defaults).

Suggested change

"editor.hover.enabled": "on"

"editor.hover.enabled": true

Copilot · 2026-02-15T14:23:46Z


 To **generate proof** use: 
-`(cd ../pil2-proofman; cargo run --release --bin proofman-cli verify-constraints --witness-lib ../zisk/target/release/libzisk_witness.so --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`
+`(cd ../pil2-proofman; cargo run --release --bin proofman-cli verify-constraints --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`


Under 'To generate proof use:', the command still runs verify-constraints (same as the previous section). This looks like a documentation error; update it to the correct proof-generation command so the two sections don't instruct the same action.

Suggested change

`(cd ../pil2-proofman; cargo run --release --bin proofman-cli verify-constraints --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`

`(cd ../pil2-proofman; cargo run --release --bin proofman-cli generate-proof --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`

Modification asm setup

…in main.c

… feature/input-1gb-allocators

…tors add new emdedded allocators, add --steps and --with-progress options …

* RefCell to Mutex * Cargo update * hints processor and asm resources refactor * Cargo update --------- Co-authored-by: RogerTaule <roger.taulee@gmail.com> Co-authored-by: Xavier Pinsach <10213118+xavi-pinsach@users.noreply.github.com> Co-authored-by: zkronos73 <zkronos73@gmail.com>

Feature/input stream merge

add partial code review report from earlier release

Copilot

Pull request overview

Copilot reviewed 151 out of 860 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-12T11:07:58Z

+    let state_u32: &mut [u32; 8] = unsafe { &mut *(state.as_mut_ptr() as *mut [u32; 8]) };
+    let input_u8: &[GenericArray<u8, U64>; 1] =
+        unsafe { &*(input.as_ptr() as *const [GenericArray<u8, U64>; 1]) };
+    compress256(state_u32, input_u8);


This changes the SHA-256 compression precompile semantics: the previous implementation explicitly split u64→u32 and produced big-endian block bytes, while this version reinterprets raw memory, making results platform-endian and swapping word order on little-endian systems. It can also violate alignment/validity assumptions for GenericArray<u8, U64>. Consider restoring explicit endian-correct conversion (or use a safe cast for [u8; 64] with defined endianness) and avoid pointer reinterprets here.

Suggested change

let state_u32: &mut [u32; 8] = unsafe { &mut *(state.as_mut_ptr() as *mut [u32; 8]) };

let input_u8: &[GenericArray<u8, U64>; 1] =

unsafe { &*(input.as_ptr() as *const [GenericArray<u8, U64>; 1]) };

compress256(state_u32, input_u8);

// Convert the 4 x u64 state into 8 x u32 in a defined (big-endian) order:

// each u64 is split into high 32 bits then low 32 bits.

let mut state_u32 = [0u32; 8];

for (i, word) in state.iter().copied().enumerate() {

state_u32[2 * i] = (word >> 32) as u32;

state_u32[2 * i + 1] = (word & 0xffff_ffff) as u32;

}

// Convert the 8 x u64 input block into 64 bytes in big-endian order.

let mut block = GenericArray::<u8, U64>::default();

for (i, word) in input.iter().copied().enumerate() {

let bytes = word.to_be_bytes();

let start = i * 8;

let end = start + 8;

block[start..end].copy_from_slice(&bytes);

}

let blocks = [block];

compress256(&mut state_u32, &blocks);

// Convert the updated 8 x u32 state back into 4 x u64 using the inverse mapping.

for i in 0..4 {

let hi = state_u32[2 * i] as u64;

let lo = state_u32[2 * i + 1] as u64;

state[i] = (hi << 32) | lo;

}

Fix pr gha inputs

…nputs add legacy-inputs options to ziskemu

Update dependencies and branch references for version 0.16.0

$@fractasy$ fractasy requested a review from Copilot February 15, 2026 14:20

cla-bot Bot added the cla-signed label Feb 15, 2026

Copilot AI reviewed Feb 15, 2026

View reviewed changes

fractasy and others added 27 commits February 23, 2026 15:19

$@fractasy$

Undo changes unrelated to asm

28480cf

Merge pull request #805 from 0xPolygonHermez/fix/ziskclib-make

69707ff

Modification asm setup

$@fractasy$

Fix riscof assembly test

deb6c79

Cleaning up mem pils

f671035

Fix book

7f916ba

First version fix

8b1ad3b

Moving Arc up

b294bef

$@fractasy$

Split assembly reset in fast/slow/trace

9790055

$@fractasy$

Fix copilot in emu.c

297d3de

$@fractasy$

Change commented code by conditional precompilation in main.c

f8896f8

$@fractasy$

Fix some comments and traces in main.c

11555c8

$@fractasy$

Synchronize shared memory areas as needed, i.e. before reading them, …

a809220

…in main.c

$@fractasy$

Redirect output to file in main.c

4d8c5f5

$@fractasy$

Enable metrics in main.c

fcd51ee

$@fractasy$

Flush Duration trace in main.c

d38d9d1

$@fractasy$

Enable verbose in main.c

a8e7e90

$@fractasy$

Add flush() to some verbose traces in main.c

30660ba

$@fractasy$

Add flush to Waiting... trace in main.c

75471cd

$@fractasy$

Flush traces before and after calling emulator_start() in main.c

4e54b4e

added traces while debugging

31ecfee

$@fractasy$

Get freopen() result in main.c

8394795

$@fractasy$

Disable traces in main.c

6a549af

adding fence in hints_shmem.rs to fix when working with hints

41f39a8

added debugging messages

24e02d7

temptative fix

7810e93

temptative fix II

76adcea

adding dbg messages

ebd4b64

zkronos73 and others added 12 commits March 11, 2026 21:19

Merge remote-tracking branch 'origin/feature/input-stream-merge' into…

3b73fc4

… feature/input-1gb-allocators

Merge pull request #842 from 0xPolygonHermez/feature/input-1gb-alloca…

82f3fa1

…tors add new emdedded allocators, add --steps and --with-progress options …

Fix reset hint size metrics

022f5c0

add code review report from earlier release

26f0f31

Adding inputs shmem stub

984af1e

Clippy fixes

1529549

More clippy fixes

70e1d73

Missing Arc Stub

341a7a6

Final fixes

9f72ba9

Merge pull request #845 from 0xPolygonHermez/feature/input-stream-merge

fe1dcc5

Feature/input stream merge

Merge pull request #844 from 0xPolygonHermez/binary-main-review

e466323

add partial code review report from earlier release

agnusmor requested a review from Copilot March 12, 2026 11:05

Copilot AI reviewed Mar 12, 2026

View reviewed changes

RogerTaule and others added 14 commits March 12, 2026 11:10

Cargo update

a58a1e1

Cargo update

e6d051e

add legacy-inputs options to ziskemu

9a7e673

Adding legacy call to convert inputs to be suitable with newer versions

fe65621

Adding option to convert full directories

019a3ea

Fix pr gha inputs

8cdb49f

Merge pull request #849 from 0xPolygonHermez/fix/gha-pr-inputs

40fa453

Fix pr gha inputs

Adding ziskup setup_snark in docs

604fefe

update hints docs

9de288d

update documentaion

dea7ffb

Merge pull request #848 from 0xPolygonHermez/feature/ziskemu-legacy-i…

2fbb38f

…nputs add legacy-inputs options to ziskemu

Update dependencies and branch references for version 0.16.0

4381288

Merge pull request #850 from 0xPolygonHermez/update-versions-0.16.0

fc82d87

Update dependencies and branch references for version 0.16.0

Update hints_stream.md to use bn254_g1_add as example

5a4a57c

agnusmor marked this pull request as ready for review March 12, 2026 16:18

agnusmor merged commit 40e35b3 into develop Mar 12, 2026
9 of 12 checks passed

-    let state_u32: &mut [u32; 8] = unsafe { &mut *(state.as_mut_ptr() as *mut [u32; 8]) };
-    let input_u8: &[GenericArray<u8, U64>; 1] =
-        unsafe { &*(input.as_ptr() as *const [GenericArray<u8, U64>; 1]) };
-    compress256(state_u32, input_u8);
+    // Convert the 4x u64 state into 8x u32 words in a fixed (big-endian) order.
+    let mut state_u32 = [0u32; 8];
+    for (i, word) in state.iter().cloned().enumerate() {
+        // High 32 bits first, then low 32 bits, independent of host endianness.
+        state_u32[2 * i] = (word >> 32) as u32;
+        state_u32[2 * i + 1] = word as u32;
+    }
+    // Convert the 8x u64 input words into a single 64-byte block in big-endian order.
+    let mut block = GenericArray::<u8, U64>::default();
+    for (i, word) in input.iter().enumerate() {
+        let bytes = word.to_be_bytes();
+        let offset = i * 8;
+        block[offset..offset + 8].copy_from_slice(&bytes);
+    }
+    // Compress a single 512-bit block.
+    compress256(&mut state_u32, std::slice::from_ref(&block));
+    // Convert the updated 8x u32 state back into 4x u64 words using the inverse mapping.
+    for i in 0..4 {
+        let hi = state_u32[2 * i] as u64;
+        let lo = state_u32[2 * i + 1] as u64;
+        state[i] = (hi << 32) | lo;
+    }

	`(cd ../pil2-proofman; cargo run --release --bin proofman-cli verify-constraints --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`
	`(cd ../pil2-proofman; cargo run --release --bin proofman-cli generate-proof --rom ../zisk/emulator/benches/data/my.elf -i ../zisk/emulator/benches/data/input.bin --proving-key ../zisk/build/provingKey)`

Conversation

fractasy commented Feb 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

$@fractasy$ fractasy commented Feb 15, 2026