Skip to content

Commit becfa36

Browse files
author
Masoud Charkhabi
committed
docs: fix README structure, update Ray version to available release
- Mark item 4 (GPU acceleration) as done - Update Ray constraint from >=2.9.0,<2.11.0 to >=2.31.0 (old versions unavailable) - Fix duplicate sections and broken structure - Simplify and clean up formatting
1 parent f395009 commit becfa36

File tree

2 files changed

+48
-53
lines changed

2 files changed

+48
-53
lines changed

README.md

Lines changed: 45 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -51,75 +51,77 @@ distributed-zkml adds:
5151
3. **Merkle Commitments**: Hash intermediate outputs with Poseidon, only root is public
5252
4. **On-Chain**: Publish only the Merkle root (O(1) public values vs O(n) without)
5353

54-
```
55-
Model: 9 layers 3 chunks
56-
Chunk 1: Layers 0-2 GPU 1 Hash A
57-
Chunk 2: Layers 3-5 GPU 2 Hash B
58-
Chunk 3: Layers 6-8 GPU 3 Hash C
54+
\`\`\`
55+
Model: 9 layers -> 3 chunks
56+
Chunk 1: Layers 0-2 -> GPU 1 -> Hash A
57+
Chunk 2: Layers 3-5 -> GPU 2 -> Hash B
58+
Chunk 3: Layers 6-8 -> GPU 3 -> Hash C
5959

6060
Merkle Tree:
6161
Root (public)
62-
/ \
62+
/ \\
6363
Hash(AB) Hash C
64-
/ \
64+
/ \\
6565
Hash A Hash B
66-
```
66+
\`\`\`
6767

6868
### Structure
6969

70-
```
70+
\`\`\`
7171
distributed-zkml/
7272
├── python/ # Python wrappers for Rust prover
7373
├── tests/ # Distributed proving tests
7474
└── zkml/ # zkml (modified for Merkle + chunking)
7575
├── src/bin/prove_chunk.rs
7676
└── testing/
77-
```
77+
\`\`\`
7878

7979
## Requirements
8080

81-
### Option 1: Docker (Recommended)
81+
### Docker (Recommended)
8282

8383
Just Docker and Docker Compose. Everything else is in the container.
8484

85-
### Option 2: Native Build
85+
### Native Build
8686

8787
| Dependency | Notes |
8888
|------------|-------|
8989
| Rust (nightly) | Install via [rustup](https://rustup.rs/) |
90-
| Python >=3.10 | 3.11 recommended on macOS x86_64 |
91-
| uv or pip | `uv sync` or `pip install -e .` |
92-
| Build tools | Linux: `build-essential pkg-config libssl-dev`; macOS: Xcode CLI |
90+
| Python >=3.10 | |
91+
| pip | \`pip install -e .\` |
92+
| Build tools | Linux: \`build-essential pkg-config libssl-dev\`; macOS: Xcode CLI |
9393

94-
**Python deps** (installed via `uv sync`):
95-
- `ray[default]>=2.9.0,<2.11.0`
96-
- `msgpack`, `numpy`
94+
**Python deps** (installed via \`pip install -e .\`):
95+
- \`ray[default]>=2.31.0\`
96+
- \`msgpack\`, \`numpy\`
9797

9898
**Optional**: NVIDIA GPU + CUDA 12.x + ICICLE backend for GPU acceleration
9999

100+
---
101+
100102
## Quick Start
101103

102104
### Docker
103105

104-
```bash
106+
\`\`\`bash
105107
docker compose build dev
106108
docker compose run --rm dev
107109
# Inside container:
108110
cd zkml && cargo test --test merkle_tree_test -- --nocapture
109-
```
111+
\`\`\`
110112

111113
### Native
112114

113-
```bash
115+
\`\`\`bash
114116
# Install Rust
115117
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
116118

117119
# Build
118120
cd zkml && rustup override set nightly && cargo build --release && cd ..
119121

120122
# Python deps
121-
uv sync # or: pip install -e .
122-
```
123+
pip install -e .
124+
\`\`\`
123125

124126
---
125127

@@ -135,9 +137,9 @@ Uses [ICICLE](https://github.com/ingonyama-zk/icicle) for GPU-accelerated MSM (M
135137

136138
### Setup
137139

138-
```bash
139-
# 1. Download ICICLE backend (Ubuntu 22.04)
140-
curl -L -o /tmp/icicle.tar.gz \
140+
\`\`\`bash
141+
# 1. Download ICICLE backend (Ubuntu 22.04 - use ubuntu20 for 20.04)
142+
curl -L -o /tmp/icicle.tar.gz \\
141143
https://github.com/ingonyama-zk/icicle/releases/download/v3.1.0/icicle_3_1_0-ubuntu22-cuda122.tar.gz
142144

143145
# 2. Install
@@ -151,7 +153,13 @@ cd zkml && cargo build --release --features gpu
151153

152154
# 5. Verify
153155
cargo test --test gpu_benchmark_test --release --features gpu -- --nocapture
154-
```
156+
\`\`\`
157+
158+
Expected output:
159+
\`\`\`
160+
Registered devices: ["CUDA", "CPU"]
161+
Successfully set CUDA device 0
162+
\`\`\`
155163

156164
### Benchmarks (Tesla T4)
157165

@@ -163,36 +171,36 @@ cargo test --test gpu_benchmark_test --release --features gpu -- --nocapture
163171

164172
### FFT/NTT Notes
165173

166-
- **Measure FFT time**: `HALO2_FFT_STATS=1`
167-
- **GPU NTT (experimental)**: `HALO2_USE_GPU_NTT=1` - currently slower due to conversion overhead
174+
- **Measure FFT time**: \`HALO2_FFT_STATS=1\`
175+
- **GPU NTT (experimental)**: \`HALO2_USE_GPU_NTT=1\` - currently slower due to conversion overhead
168176

169177
---
170178

171179
## Testing
172180

173181
### Distributed Proving
174182

175-
```bash
183+
\`\`\`bash
176184
# Simulation (fast)
177-
python tests/simple_distributed.py \
178-
--model zkml/examples/mnist/model.msgpack \
179-
--input zkml/examples/mnist/inp.msgpack \
185+
python tests/simple_distributed.py \\
186+
--model zkml/examples/mnist/model.msgpack \\
187+
--input zkml/examples/mnist/inp.msgpack \\
180188
--layers 4 --workers 2
181189

182190
# Real proofs
183191
python tests/simple_distributed.py ... --real
184-
```
192+
\`\`\`
185193

186194
### Rust Tests
187195

188-
```bash
196+
\`\`\`bash
189197
cd zkml
190198
cargo test --test merkle_tree_test --test chunk_execution_test -- --nocapture
191-
```
199+
\`\`\`
192200

193201
### CI
194202

195-
Runs on PRs to `main`/`dev`: builds zkml, runs tests (~3-4 min). GPU tests excluded to save costs.
203+
Runs on PRs to \`main\`/\`dev\`: builds zkml, runs tests (~3-4 min). GPU tests excluded to save costs.
196204

197205
---
198206

pyproject.toml

Lines changed: 3 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,14 @@ requires-python = ">=3.10"
77
license = {text = "MIT"}
88

99
dependencies = [
10-
# Ray <2.11.0 required for macOS x86_64 compatibility
11-
"ray[default]>=2.9.0,<2.11.0",
10+
"ray[default]>=2.31.0",
1211
"msgpack>=1.0.0,<2.0.0",
13-
"numpy>=1.24.0,<2.0.0",
12+
"numpy>=1.24.0,<3.0.0",
1413
]
1514

1615
[project.optional-dependencies]
1716
dev = [
18-
"pytest>=7.0.0,<8.0.0",
19-
"pytest-asyncio>=0.21.0,<1.0.0",
20-
]
21-
gpu = [
22-
"torch>=2.0.0",
17+
"pytest>=7.0.0",
2318
]
2419

2520
[build-system]
@@ -29,14 +24,6 @@ build-backend = "hatchling.build"
2924
[tool.hatch.build.targets.wheel]
3025
packages = ["python"]
3126

32-
[dependency-groups]
33-
dev = [
34-
"pytest>=7.0.0,<8.0.0",
35-
"pytest-asyncio>=0.21.0,<1.0.0",
36-
]
37-
3827
[tool.pytest.ini_options]
3928
testpaths = ["tests"]
4029
python_files = ["test_*.py", "*_test.py"]
41-
asyncio_mode = "auto"
42-

0 commit comments

Comments
 (0)