Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion perf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,5 @@ Given you have provisioned your infrastructure, you can now build and run the li
}
```
2. For a new implementation, in [`impl/Makefile` include your implementation in the `all` target.](./impl/Makefile#L7)
3. For a new version, reference version in [`runner/src/versions.ts`](./runner/src/versions.ts#L7-L43).
3. For a new version, add an entry to [`runner/versionsInput.json`](./runner/versionsInput.json) and extend the `implementation` union in [`runner/src/versions.ts`](./runner/src/versions.ts) if needed.
4. When adding a **new implementation name** (a new top-level folder under `impl/`), add that name to the `choices` list for `--test-filter` in [`runner/src/index.ts`](./runner/src/index.ts). The runner uses yargs with a fixed allow-list; omitting this causes `--test-filter <your-impl>` to fail even if the implementation appears in `versionsInput.json`.
12 changes: 9 additions & 3 deletions perf/impl/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ RUST_SUBDIRS := $(wildcard rust-libp2p/*/.)
HTTPS_SUBDIRS := $(wildcard https/*/.)
QUIC_GO_SUBDIRS := $(wildcard quic-go/*/.)
JS_SUBDIRS := $(wildcard js-libp2p/*/.)
PY_SUBDIRS := $(wildcard py-libp2p/*/.)

all: $(RUST_SUBDIRS) $(GO_SUBDIRS) $(HTTPS_SUBDIRS) $(QUIC_GO_SUBDIRS) $(JS_SUBDIRS)
all: $(RUST_SUBDIRS) $(GO_SUBDIRS) $(HTTPS_SUBDIRS) $(QUIC_GO_SUBDIRS) $(JS_SUBDIRS) $(PY_SUBDIRS)

$(RUST_SUBDIRS):
$(MAKE) -C $@
Expand All @@ -21,6 +22,9 @@ $(QUIC_GO_SUBDIRS):
$(JS_SUBDIRS):
$(MAKE) -C $@

$(PY_SUBDIRS):
$(MAKE) -C $@

go-libp2p: $(GO_SUBDIRS)

rust-libp2p: $(RUST_SUBDIRS)
Expand All @@ -31,9 +35,11 @@ quic-go: $(QUIC_GO_SUBDIRS)

js-libp2p: $(JS_SUBDIRS)

clean: $(RUST_SUBDIRS:%=%clean) $(GO_SUBDIRS:%=%clean) $(HTTPS_SUBDIRS:%=%clean) $(QUIC_GO_SUBDIRS:%=%clean) $(JS_SUBDIRS:%=%clean)
py-libp2p: $(PY_SUBDIRS)

clean: $(RUST_SUBDIRS:%=%clean) $(GO_SUBDIRS:%=%clean) $(HTTPS_SUBDIRS:%=%clean) $(QUIC_GO_SUBDIRS:%=%clean) $(JS_SUBDIRS:%=%clean) $(PY_SUBDIRS:%=%clean)

%clean:
$(MAKE) -C $* clean

.PHONY: $(RUST_SUBDIRS) $(GO_SUBDIRS) $(HTTPS_SUBDIRS) $(QUIC_GO_SUBDIRS) $(JS_SUBDIRS) all clean
.PHONY: $(RUST_SUBDIRS) $(GO_SUBDIRS) $(HTTPS_SUBDIRS) $(QUIC_GO_SUBDIRS) $(JS_SUBDIRS) $(PY_SUBDIRS) all clean
3 changes: 3 additions & 0 deletions perf/impl/py-libp2p/v0.6/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.venv/
py-libp2p-*.zip
py-libp2p-*/
31 changes: 31 additions & 0 deletions perf/impl/py-libp2p/v0.6/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Pinned py-libp2p tree (same idea as transport-interop Python image).
# Includes yamux window + perf fixes from https://github.com/libp2p/py-libp2p/pull/1258 (tip at pin time); update when merged to main.
commitSha := 731c3c787d3e98e380263943132d301c1be8dcd5

# Use 3.11–3.12 for libp2p deps (e.g. coincurve wheels); avoid 3.14+ where builds fail.
PYTHON_FOR_VENV ?= python3.12

LIBP2P_DIR := py-libp2p-$(commitSha)
VENV := .venv
PYTHON := $(VENV)/bin/python
PIP := $(VENV)/bin/pip

all: $(PYTHON) perf_cli.py
chmod +x perf

$(LIBP2P_DIR).zip:
wget -O $@ "https://github.com/libp2p/py-libp2p/archive/$(commitSha).zip"

$(LIBP2P_DIR): $(LIBP2P_DIR).zip
unzip -o $<
touch $@

$(PYTHON): $(LIBP2P_DIR)
$(PYTHON_FOR_VENV) -m venv $(VENV)
$(PIP) install --upgrade pip
$(PIP) install "./$(LIBP2P_DIR)/"

clean:
rm -rf $(VENV) $(LIBP2P_DIR).zip $(LIBP2P_DIR)

.PHONY: all clean
33 changes: 33 additions & 0 deletions perf/impl/py-libp2p/v0.6/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# py-libp2p perf (test-plans)

This directory implements the `perf` executable used by the perf benchmark runner (CLI flags + JSON on stdout; see [`perf/README.md`](../../README.md)). It is **not** the unified-testing harness in the py-libp2p repo ([`interop/perf/perf_test.py`](https://github.com/libp2p/py-libp2p/blob/main/interop/perf/perf_test.py), Redis/YAML): that flow does not match this runner’s contract.

## Build

```bash
make # downloads pinned py-libp2p, creates .venv, installs libp2p
```

Requires `python3.12` (or set `PYTHON_FOR_VENV`, e.g. `python3.11`) so dependencies such as `coincurve` install cleanly.

## Pinned `libp2p` revision

The `commitSha` in [`Makefile`](./Makefile) selects the GitHub archive used for `pip install`. It is currently aligned with [PR #1258](https://github.com/libp2p/py-libp2p/pull/1258) (yamux receive-window and related perf fixes). After that PR merges into `main`, update the pin to a stable `main` commit.

## Implementation notes

- **`perf_cli.py`** uses `PerfService(..., {"write_block_size": 65500})` so writes stay under Noise’s **65535-byte** frame limit; the library default block size can break runner-style upload throughput.
- **Server:** listens on **TCP** (Noise + Yamux) and **QUIC** on the same port pattern (two internal listeners), with the same deterministic peer id as the Go reference perf binary.
- **QUIC client — half-close:** `NetStream.close_write()` in py-libp2p calls `muxed_stream.close()`, which on `QUICStream` closes *both* halves of the stream; the client then cannot read the download phase. `perf_cli.py` patches `NetStream.close_write` to call `close_write()` on the muxed stream when available (upstream: `libp2p/network/stream/net_stream.py`).
- **QUIC client — identify:** `BasicHost` schedules background identify only for QUIC dialers, which opens a second stream on the same connection as perf. The peerstore is seeded with a cached “safe” protocol so identify is skipped (same effect as `BasicHost._has_cached_protocols`).
- **QUIC — aioquic flow limits:** aioquic defaults `QuicConfiguration.max_data` / `max_stream_data` to **1 MiB**. py-libp2p’s `QUICTransportConfig.CONNECTION_FLOW_CONTROL_WINDOW` / `STREAM_FLOW_CONTROL_WINDOW` are not applied to those fields. `perf_cli.py` wraps `create_server_config_from_base` / `create_client_config_from_base` in **both** `libp2p.transport.quic.utils` and `libp2p.transport.quic.transport` (the transport module re-imports the names, so patching only `utils` is not enough) and sets `max_data` / `max_stream_data` from the transport config. **`QUICTransportConfig`** is also passed into `new_host` for QUIC server and client with larger windows and interop-style timeouts.
- **Throughput asymmetry:** With the above, **QUIC upload** in long `timeout …` runs is typically in the same ballpark as TCP on loopback; **QUIC download** (server → client) may still be much lower than TCP in local tests — remaining limits likely involve py-libp2p/aioquic receive scheduling rather than the 1 MiB defaults alone (bumping `max_data` further did not change download totals in quick A/B tests).
- **Runner matrix:** [`runner/versionsInput.json`](../../runner/versionsInput.json) includes **`tcp`** and **`quic-v1`** for this implementation.

## Smoke (local)

```bash
./perf --run-server --server-address 127.0.0.1:4001 &
./perf --server-address 127.0.0.1:4001 --transport tcp --upload-bytes 4096 --download-bytes 4096
./perf --server-address 127.0.0.1:4001 --transport quic-v1 --upload-bytes 4096 --download-bytes 4096
```
4 changes: 4 additions & 0 deletions perf/impl/py-libp2p/v0.6/perf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$DIR/.venv/bin/python" "$DIR/perf_cli.py" "$@"
Loading