Skip to content

Commit 1592d66

Browse files
committed
chore: fixed all linter errors
1 parent 1df74e2 commit 1592d66

27 files changed

+462
-66
lines changed

docs/examples.metrics.rst

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
Metrics Demo
2+
============
3+
4+
This example demonstrates how to run multiple libp2p services (Ping, Pubsub/Gossipsub, Kad-DHT) in a single node and observer
5+
their behaviour through Prometheus + Grafana metrics dashboards.
6+
7+
.. code-block:: console
8+
9+
$ python -m pip install libp2p
10+
Collecting libp2p
11+
...
12+
Successfully installed libp2p-x.x.x
13+
14+
$ metrics-demo
15+
Host multiaddr: /ip4/172.16.68.73/tcp/41173/p2p/12D3KooWD2DFvDs4wekLWU8sAUJJgivbRbiiKkX9yQ3kGhuCwCqL
16+
Gossipsub and Pubsub services started !!
17+
DHT service started with DHTMode.SERVER mode
18+
Starting command executor loop...
19+
20+
Prometheus metrics visible at: http://localhost:8000
21+
22+
To start prometheus and grafana dashboards, from another terminal:
23+
PROMETHEUS_PORT=9001 GRAFANA_PORT=7001 docker compose up
24+
25+
After this:
26+
Prometheus dashboard will be visible at: http://localhost:9001
27+
Grafana dashboard will be visible at: http://localhost:7001
28+
29+
Entering intractive mode, type commands below.
30+
31+
Available commands:
32+
- connect <multiaddr> - Connect to another peer
33+
...
34+
35+
Now in this way a node can be started, now start another node in a different terminal
36+
and make a connection between so that they can communicate:
37+
38+
.. code-block:: console
39+
40+
$ metrics-demo
41+
$ connect /ip4/172.16.68.73/tcp/41173/p2p/12D3KooWD2DFvDs4wekLWU8sAUJJgivbRbiiKkX9yQ3kGhuCwCqL
42+
Connected to 12D3KooWD2DFvDs4wekLWU8sAUJJgivbRbiiKkX9yQ3kGhuCwCqL
43+
44+
Now we can communicate between the 2 nodes via Ping, Gossipsub and Kad-DHT. Before that we have to
45+
start the prometheus and grafana dashboards. For this create a `docker-compose.yml` file like this:
46+
47+
.. code-block:: console
48+
49+
services:
50+
prometheus:
51+
image: prom/prometheus:latest
52+
container_name: prometheus
53+
volumes:
54+
- ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
55+
ports:
56+
- "${PROMETHEUS_PORT}:9090"
57+
extra_hosts:
58+
- "host.docker.internal:host-gateway"
59+
60+
grafana:
61+
image: grafana/grafana:latest
62+
container_name: grafana
63+
ports:
64+
- "${GRAFANA_PORT}:3000"
65+
depends_on:
66+
- prometheus
67+
68+
And run it like this
69+
.. code-block:: console
70+
71+
PROMETHEUS_PORT=9001 GRAFANA_PORT=7001 docker compose up
72+
73+
A similar file is present in `py-libp2p/libp2p/metrics` directory also, so either create a new docker-compose
74+
file or run it from the above path. This basically starts a prometheus and grafana server in your localhost,
75+
with which the metrics can be viewed in graph format.
76+
77+
Now see how to communicate between the 2 nodes, via Pubsub/Gossipsub, Ping and Kad-DHT
78+
79+
PING
80+
====
81+
82+
The following metrics are exposed in this service:
83+
- ping: Round-trip time sending a `ping` and receiving a `pong`
84+
- ping_failure: Failure while sending a ping or receiving a ping
85+
86+
.. code-block:: console
87+
88+
$ ping /ip4/172.16.68.73/tcp/41173/p2p/12D3KooWD2DFvDs4wekLWU8sAUJJgivbRbiiKkX9yQ3kGhuCwCqL 15
89+
[401, 419, 428, 353, 354, 353, 369, 371, 353, 380, 352, 343, 378, 324, 412]
90+
91+
The output will the rtts took for each ping/ping to complete.
92+
The updated metrics can be visualized in the dashboards.
93+
94+
Pubsub/Gossipsub
95+
================
96+
97+
The following metrics are exposed in this service:
98+
- gossipsub_received_total: Messages successfully received
99+
- gossipsub_publish_total: Messages to be published
100+
- gossipsub_subopts_total: Messages notifying peer subscriptions
101+
- gossipsub_control_total: Received control messages
102+
- gossipsub_message_bytes: Message size in bytes
103+
104+
To communicate via gossipsub, join the same topics on both the nodes and publish messages
105+
on that topic to get it received on both sides.
106+
107+
.. code-block:: console
108+
109+
$ join pubsub-chat
110+
Subscribed to pubsub-chat
111+
Starting receive loop
112+
113+
Do this on both the terminals. Then publish a message from one side, and see it recieved on the other side.
114+
115+
.. code-block:: console
116+
117+
$ publish pubsub-chat hello-from-pubsub!
118+
119+
See the updated metrics in the dashboards.
120+
121+
KAD-DHT
122+
=======
123+
124+
The following metrics are exposed in this service:
125+
- kad_inbound_total: Total inbound requests received
126+
- kad_inbound_find_node: Total inbound FIND_NODE requests received
127+
- kad_inbound_get_value: Total inbound GET_VALUE requests received
128+
- kad_inbound_put_value: Total inbound PUT_VALUE requests received
129+
- kad_inbound_get_providers: Total inbound GET_PROVIDERS requests received
130+
- kad_inbound_add_provider: Total inbound ADD_PROVIDER requests received
131+
132+
To intercat between the 2 nodes via kad-dht, we have 2 ways:
133+
- `PUT_VAUE` in one node, and `GET_VALUE` in another
134+
- `ADD_PROVIDER` in one node, and `GET_PROVIDERS` in another
135+
136+
.. code-block:: console
137+
138+
$ put /exp/fa kad-dht-value
139+
Stored value: kad-dht-value with key: /exp/fa
140+
141+
# From another terminal
142+
$ get /exp/fa
143+
Retrieved value: kad-dht-value
144+
145+
.. code-block:: console
146+
147+
$ advertize content-id
148+
Advertised as provider for content: content-id
149+
150+
# From another terminal
151+
$ get_provider content-id
152+
Found 1 providers: [<libp2p.peer.id.ID (12D3KooWD2DFvDs4wekLWU8sAUJJgivbRbiiKkX9yQ3kGhuCwCqL)>]
153+
154+
SWARM-CONNECTION-EVENTS
155+
=======================
156+
157+
Other than the above 3 services, the incoming/outgoing connection cycle is also monitored via the
158+
following metrics:
159+
- swarm_incoming_conn: Incoming connection received by libp2p-swarm
160+
- swarm_incoming_conn_error: Incoming connection failure in libp2p-swarm
161+
- swarm_dial_attempt: Dial attempts made by libp2p-swarm
162+
- swarm_dial_attempt_error: Outgoing connection failure in libp2p-swarm
163+
164+
The full source code for this example is below:
165+
166+
.. literalinclude:: ../examples/metrics/runner.py
167+
:language: python
168+
:linenos:

docs/examples.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,4 @@ Examples
2929
examples.autotls
3030
examples.perf
3131
examples.path_handling
32+
examples.metrics

docs/libp2p.metrics.rst

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
libp2p.metrics package
2+
======================
3+
4+
Submodules
5+
----------
6+
7+
libp2p.metrics.gossipsub module
8+
-------------------------------
9+
10+
.. automodule:: libp2p.metrics.gossipsub
11+
:members:
12+
:undoc-members:
13+
:show-inheritance:
14+
15+
libp2p.metrics.kad_dht module
16+
-----------------------------
17+
18+
.. automodule:: libp2p.metrics.kad_dht
19+
:members:
20+
:undoc-members:
21+
:show-inheritance:
22+
23+
libp2p.metrics.metrics module
24+
-----------------------------
25+
26+
.. automodule:: libp2p.metrics.metrics
27+
:members:
28+
:undoc-members:
29+
:show-inheritance:
30+
31+
libp2p.metrics.ping module
32+
--------------------------
33+
34+
.. automodule:: libp2p.metrics.ping
35+
:members:
36+
:undoc-members:
37+
:show-inheritance:
38+
39+
libp2p.metrics.swarm module
40+
---------------------------
41+
42+
.. automodule:: libp2p.metrics.swarm
43+
:members:
44+
:undoc-members:
45+
:show-inheritance:
46+
47+
Module contents
48+
---------------
49+
50+
.. automodule:: libp2p.metrics
51+
:members:
52+
:undoc-members:
53+
:show-inheritance:

docs/libp2p.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ Subpackages
2828
libp2p.tools
2929
libp2p.transport
3030
libp2p.utils
31+
libp2p.metrics
3132

3233
Submodules
3334
----------

examples/kademlia/server_node_addr.txt

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)