Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
fe89373
the first version of the perf protocol
ItshMoh Jan 28, 2026
ba19233
fixed the lint errors
ItshMoh Jan 28, 2026
a9d4848
changed the description
ItshMoh Jan 29, 2026
e0238b1
Merge branch 'main' into perfexamples
seetadev Jan 29, 2026
d977f50
fixed the yamux and noise header size errors
ItshMoh Jan 31, 2026
98df9ab
Merge branch 'perfexamples' of github.com:ItshMoh/py-libp2p into perf…
ItshMoh Jan 31, 2026
2bff259
wip: first interop perf experiment
acul71 Feb 1, 2026
5eded13
fix: Dockerfile as transport
acul71 Feb 1, 2026
99741f2
Merge branch 'main' into perfexamples
acul71 Feb 1, 2026
2c48b4c
fixed the lint issue
ItshMoh Feb 2, 2026
195f711
moved the interfaces to abc.py file and restructure the code
ItshMoh Feb 2, 2026
efdb1cd
pushed the .rst files
ItshMoh Feb 2, 2026
814bdda
Merge branch 'main' of github.com:ItshMoh/py-libp2p into perfexamples
ItshMoh Feb 2, 2026
25e7a76
included the perf in examples.rst
ItshMoh Feb 2, 2026
8d3a784
Merge branch 'main' into perfexamples
acul71 Feb 3, 2026
10d359d
pushed the newsfragments
ItshMoh Feb 3, 2026
7fdb852
fixed thelength issue
ItshMoh Feb 3, 2026
cd80d31
Merge branch 'main' of github.com:ItshMoh/py-libp2p into perfexamples
ItshMoh Feb 3, 2026
e75e432
Merge branch 'perfexamples' of github.com:ItshMoh/py-libp2p into perf…
ItshMoh Feb 3, 2026
80e2b96
fixed some issues in perf_test.py
ItshMoh Feb 3, 2026
a6b029a
fixed the 8bytes logic error
ItshMoh Feb 3, 2026
dad99d4
tests for the perf
ItshMoh Feb 3, 2026
3903a99
Merge branch 'main' into perfexamples
seetadev Feb 5, 2026
1f2f795
Merge branch 'main' of github.com:ItshMoh/py-libp2p into perfexamples
ItshMoh Feb 9, 2026
c0298da
changes the newsfragment name to the issue number
ItshMoh Feb 9, 2026
6a7767f
Merge branch 'perfexamples' of github.com:ItshMoh/py-libp2p into perf…
ItshMoh Feb 9, 2026
a4b5c28
fixed the lint errors in init
ItshMoh Feb 9, 2026
ccb0c58
added the docstring about the high data to send back and resouce exha…
ItshMoh Feb 9, 2026
4fefe15
Merge branch 'main' into perfexamples
seetadev Feb 9, 2026
d739255
fixed the perfoptions and header type issue
ItshMoh Feb 10, 2026
8493989
fixed the lint issue by changing the type of chunk
ItshMoh Feb 12, 2026
d0cbd22
fixed teh lint errors by the ignore
ItshMoh Feb 12, 2026
c6c7e29
Merge branch 'main' into perfexamples
acul71 Feb 14, 2026
62c7542
Merge branch 'main' of github.com:ItshMoh/py-libp2p into perfexamples
ItshMoh Feb 16, 2026
2e3fd23
Merge branch 'perfexamples' of github.com:ItshMoh/py-libp2p into perf…
ItshMoh Feb 16, 2026
08372ca
used the .started variable in the handle message function
ItshMoh Feb 16, 2026
a2a922a
removed the configs that are not implemented in the set_stream_handle…
ItshMoh Feb 16, 2026
bba4e2a
Merge branch 'main' into perfexamples
seetadev Feb 16, 2026
578ee35
Merge branch 'main' into perfexamples
seetadev Feb 16, 2026
8fd2f3a
Merge branch 'main' into perfexamples
acul71 Feb 17, 2026
5a306b3
fixed the typo
ItshMoh Feb 17, 2026
d963285
Merge branch 'perfexamples' of github.com:ItshMoh/py-libp2p into perf…
ItshMoh Feb 17, 2026
5aecd09
removed the config option detail from the newsfragment
ItshMoh Feb 17, 2026
842714d
Merge branch 'main' into perfexamples
acul71 Feb 18, 2026
6bc8a23
Merge branch 'main' into perfexamples
acul71 Feb 25, 2026
a5acc03
fix: resolve perf interop addr rebuild and align block size
acul71 Feb 25, 2026
e8856b2
Merge branch 'main' into perfexamples
acul71 Feb 28, 2026
b43d971
doc: interop/ dir README.md for unified-test new repo
acul71 Feb 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/examples.perf.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
Perf Protocol Demo
==================

This example demonstrates how to use the libp2p ``perf`` protocol to measure
transfer performance between two nodes.

The perf protocol sends and receives data to measure throughput, reporting
both intermediary progress and final results.

Running the Example
-------------------

First, start the server in one terminal:

.. code-block:: console

$ python examples/perf/perf_example.py -p 8000

Perf server ready, listening on:
/ip4/127.0.0.1/tcp/8000/p2p/QmXfptdHU6hqG95JswxYVUH4bphcK8y18mhFcgUQFe6fCN

Protocol: /perf/1.0.0

Run client with:
python perf_example.py -d /ip4/127.0.0.1/tcp/8000/p2p/QmXfptdHU6hqG95JswxYVUH4bphcK8y18mhFcgUQFe6fCN

Waiting for incoming perf requests...

Then, in another terminal, run the client with the multiaddr from the server:

.. code-block:: console

$ python examples/perf/perf_example.py -d /ip4/127.0.0.1/tcp/8000/p2p/QmXfptdHU6hqG95JswxYVUH4bphcK8y18mhFcgUQFe6fCN

Connecting to QmXfptdHU6hqG95JswxYVUH4bphcK8y18mhFcgUQFe6fCN...
Connected!

Measuring performance:
Upload: 2560 bytes
Download: 2560 bytes

Uploading: 2560 bytes in 0.01s (256000 bytes/s)
Downloading: 2560 bytes in 0.01s (256000 bytes/s)

==================================================
Performance Results:
Total time: 0.025 seconds
Uploaded: 2560 bytes
Downloaded: 2560 bytes
Total data: 5120 bytes
Throughput: 204800 bytes/s
==================================================

Command Line Options
--------------------

.. code-block:: text

-p, --port Listening port (default: random free port)
-d, --destination Destination multiaddr (if not set, runs as server)
-u, --upload Upload size in units of 256 bytes (default: 10)
-D, --download Download size in units of 256 bytes (default: 10)

Source Code
-----------

.. literalinclude:: ../examples/perf/perf_example.py
:language: python
:linenos:
1 change: 1 addition & 0 deletions docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ Examples
examples.websocket
examples.tls
examples.autotls
examples.perf
41 changes: 41 additions & 0 deletions docs/libp2p.perf.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
libp2p.perf package
===================

The perf module implements the libp2p performance measurement protocol,
which allows measuring throughput between two libp2p nodes by sending
and receiving configurable amounts of data.

Submodules
----------

libp2p.perf.constants module
----------------------------

.. automodule:: libp2p.perf.constants
:members:
:undoc-members:
:show-inheritance:

libp2p.perf.types module
------------------------

.. automodule:: libp2p.perf.types
:members:
:undoc-members:
:show-inheritance:

libp2p.perf.perf\_service module
--------------------------------

.. automodule:: libp2p.perf.perf_service
:members:
:undoc-members:
:show-inheritance:

Module contents
---------------

.. automodule:: libp2p.perf
:members:
:undoc-members:
:show-inheritance:
1 change: 1 addition & 0 deletions docs/libp2p.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Subpackages
libp2p.kad_dht
libp2p.network
libp2p.peer
libp2p.perf
libp2p.protocol_muxer
libp2p.pubsub
libp2p.rcmgr
Expand Down
165 changes: 165 additions & 0 deletions examples/perf/perf_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
"""
Perf protocol example - Measure transfer performance between two libp2p nodes.

Usage:
# Terminal 1 - Run the server (listener)
python perf_example.py -p 8000

# Terminal 2 - Run the client (measures performance to server)
python perf_example.py -p 8001 -d /ip4/127.0.0.1/tcp/8000/p2p/<PEER_ID>
"""

import argparse

import multiaddr
import trio

from libp2p import new_host
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.perf import PROTOCOL_NAME, PerfService

ONE_UNIT = 16 * 16 # 256 bytes
UPLOAD_BYTES = ONE_UNIT * 10 # 2560 bytes upload
DOWNLOAD_BYTES = ONE_UNIT * 10 # 2560 bytes download


async def run_server(host, perf_service) -> None:
"""Run as a perf server - listens for incoming perf requests."""
await perf_service.start()

print("\nPerf server ready, listening on:")
for addr in host.get_addrs():
print(f" {addr}")

print(f"\nProtocol: {PROTOCOL_NAME}")
print("\nRun client with:")
print(f" python perf_example.py -d {host.get_addrs()[0]}")
print("\nWaiting for incoming perf requests...")

await trio.sleep_forever()


async def run_client(
host, perf_service, destination: str, upload_bytes: int, download_bytes: int
) -> None:
"""Run as a perf client - measures performance to a remote peer."""
await perf_service.start()

maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)

print(f"\nConnecting to {info.peer_id}...")
await host.connect(info)
print("Connected!")

print("\nMeasuring performance:")
print(f" Upload: {upload_bytes} bytes")
print(f" Download: {download_bytes} bytes")
print()

async for output in perf_service.measure_performance(
maddr, upload_bytes, download_bytes
):
if output["type"] == "intermediary":
# Progress report
upload_bytes_out = output["upload_bytes"]
download_bytes_out = output["download_bytes"]
time_s = output["time_seconds"]

if upload_bytes_out > 0:
throughput = upload_bytes_out / time_s if time_s > 0 else 0
print(
f" Upload: {upload_bytes_out}B in {time_s:.2f}s "
f"({throughput:.0f} B/s)"
)
elif download_bytes_out > 0:
throughput = download_bytes_out / time_s if time_s > 0 else 0
print(
f" Download: {download_bytes_out}B in {time_s:.2f}s "
f"({throughput:.0f} B/s)"
)

elif output["type"] == "final":
# Final summary
total_time = output["time_seconds"]
total_upload = output["upload_bytes"]
total_download = output["download_bytes"]
total_data = total_upload + total_download

print(f"\n{'=' * 50}")
print("Performance Results:")
print(f" Total time: {total_time:.3f} seconds")
print(f" Uploaded: {total_upload} bytes")
print(f" Downloaded: {total_download} bytes")
print(f" Total data: {total_data} bytes")
print(f" Throughput: {total_data / total_time:.0f} bytes/s")
print(f"{'=' * 50}")

await perf_service.stop()


async def run(port: int, destination: str, upload_mb: int, download_mb: int) -> None:
"""Main run function."""
from libp2p.utils.address_validation import find_free_port

if port <= 0:
port = find_free_port()

listen_addrs = [multiaddr.Multiaddr(f"/ip4/127.0.0.1/tcp/{port}")]
host = new_host(listen_addrs=listen_addrs)

# Create perf service
perf_service = PerfService(host)

async with host.run(listen_addrs=listen_addrs):
if destination:
# Client mode
await run_client(
host,
perf_service,
destination,
upload_mb * ONE_UNIT,
download_mb * ONE_UNIT,
)
else:
# Server mode
await run_server(host, perf_service)


def main() -> None:
description = """
Perf protocol example - Measure transfer performance between libp2p nodes.

To use:
1. Start server: python perf_example.py -p 8000
2. Start client: python perf_example.py -d <MULTIADDR_FROM_SERVER>
"""

parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="listening port")
parser.add_argument("-d", "--destination", type=str, help="destination multiaddr")
parser.add_argument(
"-u",
"--upload",
default=10,
type=int,
help="upload size in units of 256 bytes (default: 10)",
)
parser.add_argument(
"-D",
"--download",
default=10,
type=int,
help="download size in units of 256 bytes (default: 10)",
)

args = parser.parse_args()

try:
trio.run(run, args.port, args.destination, args.upload, args.download)
except KeyboardInterrupt:
print("\nShutting down...")


if __name__ == "__main__":
main()
86 changes: 86 additions & 0 deletions interop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Interop Directory

This directory contains **interoperability and performance test material** for py-libp2p, designed to integrate with the [unified-testing framework](https://github.com/libp2p/unified-testing) (successor to libp2p/test-plans).

## Purpose

The `interop/` directory houses the Docker images, test scripts, and configuration needed to run py-libp2p in cross-implementation tests. The unified-testing convention allows this material to live in the implementation repository itself, which:

- Enables **local testing** against other libp2p implementations (Rust, Go, JS, .NET) without syncing between repos
- Serves as an **example** for setting up Docker-based protocol tests with py-libp2p
- Keeps test implementations versioned and developed alongside the library

## Directory Layout

```
interop/
├── perf/ # Performance (throughput, latency) tests
│ ├── Dockerfile # Builds image for perf protocol testing
│ ├── perf_test.py # Test application (listener + dialer)
│ └── pyproject.toml # Dependencies (libp2p, redis, etc.)
└── transport/ # Transport interoperability tests
├── Dockerfile
├── ping_test.py
└── pyproject.toml
```

Each subdirectory corresponds to a **test type** in the unified-testing framework.

## How It Integrates with Unified-Testing

The unified-testing framework (see `unified-testing/docs/`) runs tests by:

1. **Building Docker images** – Uses `images.yaml` to define implementations. For py-libp2p, the build uses `source.type: github` (or `local`) pointing at this repo, with a `dockerfile` path such as `interop/perf/Dockerfile`.
1. **Running tests** – The framework starts **listener** and **dialer** containers on a shared network, coordinates them via **Redis**, and collects results.

### Perf Tests (`interop/perf/`)

Perf tests measure:

- **Upload throughput** – How fast the dialer sends data to the listener
- **Download throughput** – How fast the dialer receives data from the listener
- **Latency** – Round-trip time for small messages

The test app (`perf_test.py`) implements the [libp2p perf protocol](https://github.com/libp2p/specs/blob/master/perf/perf.md) (`/perf/1.0.0`) and follows [write-a-perf-test-app.md](https://github.com/libp2p/unified-testing/blob/master/docs/write-a-perf-test-app.md):

- Reads config from environment variables (`IS_DIALER`, `REDIS_ADDR`, `TEST_KEY`, `TRANSPORT`, etc.)
- Listener publishes its multiaddr to Redis; dialer polls and connects
- Dialer runs upload/download/latency iterations and outputs YAML results to stdout
- All logging goes to stderr (stdout is reserved for results)

### Transport Tests (`interop/transport/`)

Transport tests verify that py-libp2p can establish connections and exchange protocols with other implementations over various transport, secure channel, and muxer combinations (TCP, QUIC, WebSocket, Noise, TLS, yamux, mplex).

## Build Context

When building from the py-libp2p repo:

- **Build context** = repository root (not `interop/perf/` or `interop/transport/`)
- The Dockerfile uses `COPY . /app/py-libp2p` to include the full libp2p source, then copies the test script and installs dependencies so the test app uses the in-repo libp2p.

This ensures each Docker image is built against the exact py-libp2p version in the repo or specified commit.

## Running Locally

To run interop tests, use the unified-testing framework:

```bash
# From the unified-testing repo
cd perf
./run.sh --impl-select "python-v0.x" # When python is in images.yaml
```

To run the perf test script directly (e.g. for development), see `examples/perf/perf_example.py` and the [perf protocol documentation](../../libp2p/perf/).

## Relationship to CI

- Code in `interop/` is **not** run by py-libp2p's own CI (which uses `tests/`).
- The unified-testing framework runs this code when py-libp2p is included in `images.yaml` and the perf/transport test suite is executed (e.g. in the test-plans or unified-testing repo).

## References

- [Unified-testing framework](https://github.com/libp2p/unified-testing) – Bash + Docker test runner
- [write-a-perf-test-app.md](https://github.com/libp2p/unified-testing/blob/master/docs/write-a-perf-test-app.md) – Perf test app specification
- [write-a-transport-test-app.md](https://github.com/libp2p/unified-testing/blob/master/docs/write-a-transport-test-app.md) – Transport test app specification
- [libp2p perf protocol spec](https://github.com/libp2p/specs/blob/master/perf/perf.md)
Loading
Loading