Skip to content

Commit 6b0d806

Browse files
lucassaldanhalucassaldanha
committed
Added claude config
1 parent c2554a6 commit 6b0d806

2 files changed

Lines changed: 319 additions & 1 deletion

File tree

‎.gitignore‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -192,4 +192,8 @@ package-lock.json
192192
/src/jmh/java/generated/
193193

194194
#Jenv
195-
.java-version
195+
.java-version
196+
197+
# Claude
198+
CLAUDE.local.md
199+
.claude/settings.local.json

‎CLAUDE.md‎

Lines changed: 314 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,314 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
jvm-libp2p is a JVM implementation of the [libp2p](https://libp2p.io/) networking stack, written in Kotlin. It provides peer-to-peer networking capabilities including transport protocols (TCP, QUIC, WebSocket), security channels (Noise, TLS), stream multiplexing (Yamux, Mplex), and pub/sub messaging (Gossipsub, Floodsub).
8+
9+
Notable users: Teku (Ethereum Consensus Layer client), Nabu (minimal IPFS), Peergos (peer-to-peer encrypted filesystem).
10+
11+
## Build Commands
12+
13+
```bash
14+
# Build the entire project
15+
./gradlew build
16+
17+
# Run all tests (excludes interop tests tagged with "interop")
18+
./gradlew test
19+
20+
# Run tests for a specific module
21+
./gradlew :libp2p:test
22+
23+
# Run a specific test class
24+
./gradlew :libp2p:test --tests "io.libp2p.pubsub.gossip.GossipRpcPartsQueueTest"
25+
26+
# Run a specific test method
27+
./gradlew :libp2p:test --tests "io.libp2p.pubsub.gossip.GossipRpcPartsQueueTest.mergeMessageParts*"
28+
29+
# Check code formatting
30+
./gradlew spotlessCheck
31+
32+
# Apply code formatting
33+
./gradlew spotlessApply
34+
35+
# Run static analysis (Detekt)
36+
./gradlew detekt
37+
38+
# Generate documentation
39+
./gradlew dokkaHtml
40+
# Output in build/dokka/
41+
42+
# Clean build artifacts
43+
./gradlew clean
44+
```
45+
46+
**Requirements:** JDK 11 or higher
47+
48+
**Module Structure:**
49+
- `:libp2p` - Main library module
50+
- `:tools:simulator` - Gossip network simulator
51+
- `:tools:schedulers` - Test scheduling utilities
52+
- `:examples:chatter`, `:examples:cli-chatter`, `:examples:pinger` - Example applications
53+
- `:interop-test-client` - Interoperability testing client
54+
55+
## Architecture Overview
56+
57+
### Core Abstraction Layers
58+
59+
The library follows a layered architecture with protocol negotiation at each layer:
60+
61+
```
62+
Application Layer
63+
↓ (Protocol negotiation via multistream-select)
64+
Stream/Protocol Layer (PingProtocol, ChatProtocol, PubsubRouter)
65+
↓ (Stream creation)
66+
Stream Multiplexing Layer (Yamux, Mplex)
67+
↓ (Multiplexer negotiation)
68+
Security Layer (Noise, TLS)
69+
↓ (Security negotiation)
70+
Transport Layer (TCP, QUIC, WebSocket)
71+
↓
72+
Raw Network
73+
```
74+
75+
### Key Interfaces and Their Roles
76+
77+
**`Host`** (`core/Host.kt`):
78+
- Main entry point for all libp2p operations
79+
- Manages identity (`PeerId`, `PrivKey`), network, and protocol handlers
80+
- Created via DSL builder: `host { identity { ... }; transports { ... }; protocols { ... } }`
81+
82+
**`Network`** (`core/Network.kt`):
83+
- Manages transports and active connections
84+
- Handles `listen()` and `dial()` operations
85+
- Reuses connections to the same peer
86+
87+
**`Connection`** and **`Stream`** (both extend `P2PChannel`):
88+
- `Connection`: Secured, multiplexed connection between two peers
89+
- `Stream`: Logical stream over a connection for a specific protocol
90+
91+
**`Transport`** (`transport/Transport.kt`):
92+
- Handles raw connection establishment (TCP, QUIC, WebSocket)
93+
- Each transport parses specific multiaddr formats (e.g., `/ip4/127.0.0.1/tcp/30333`)
94+
95+
**`SecureChannel`** (`security/SecureChannel.kt`):
96+
- Protocol binding for security layer negotiation
97+
- Returns `SecureChannel.Session` with `remoteId`, `remotePubKey`
98+
- Implementations: `NoiseXXSecureChannel` (production), `TlsSecureChannel` (beta)
99+
100+
**`StreamMuxer`** (`mux/StreamMuxer.kt`):
101+
- Protocol binding for multiplexer negotiation
102+
- Returns `StreamMuxer.Session` for creating/receiving streams
103+
- Implementations: `MplexStreamMuxer` (production), `YamuxStreamMuxer` (beta)
104+
105+
### The Connection Upgrade Pipeline
106+
107+
When a raw transport connection is established, it goes through staged upgrades:
108+
109+
```
110+
1. Raw Transport (TCP/QUIC/WS)
111+
↓
112+
2. ConnectionBuilder (transport/implementation/ConnectionBuilder.kt)
113+
↓
114+
3. Security Negotiation → SecureChannel.Session
115+
↓
116+
4. Multiplexer Negotiation → StreamMuxer.Session
117+
↓
118+
5. Full Connection Ready → ConnectionOverNetty
119+
```
120+
121+
**Key Class:** `ConnectionUpgrader` (`transport/implementation/ConnectionUpgrader.kt`)
122+
- Orchestrates security and muxer protocol negotiation
123+
- Uses `MultistreamProtocol` for protocol selection
124+
- Supports early muxer negotiation (TLS 1.3 feature)
125+
126+
### Protocol Handler Pattern
127+
128+
Custom protocols implement `ProtocolHandler<TController>`:
129+
130+
```kotlin
131+
// Define protocol binding
132+
StrictProtocolBinding("/ipfs/ping/1.0.0", PingProtocol())
133+
134+
// Implement handler
135+
class PingProtocol : ProtocolHandler<PingController> {
136+
override fun onStartInitiator(stream: Stream): CompletableFuture<PingController>
137+
override fun onStartResponder(stream: Stream): CompletableFuture<PingController>
138+
}
139+
```
140+
141+
See `examples/chatter/ChatProtocol.kt` for a complete example.
142+
143+
### Pub/Sub Architecture
144+
145+
The pub/sub system is located in `pubsub/` and follows this structure:
146+
147+
**`AbstractRouter`** (`pubsub/AbstractRouter.kt`):
148+
- Base class providing common pubsub logic
149+
- Manages peer subscriptions via `peersTopics` (multi-bimap)
150+
- Implements message validation, deduplication (via `SeenCache`), and batching
151+
- Uses single-threaded event loop (`P2PService`) for thread-safety
152+
153+
**Message Batching via `RpcPartsQueue`**:
154+
- Per-peer queue that accumulates message parts before transmission
155+
- Pattern: accumulate parts → flush via `takeMerged()` → send merged RPC
156+
- Default implementation merges all parts into single RPC
157+
- Gossip implementation (`GossipRpcPartsQueue`) splits messages to respect per-category limits
158+
159+
**Message Flow:**
160+
```
161+
Outbound: publish() → validateAndBroadcast() → submitPublishMessage(peer)
162+
→ queue.addPublish() → flushPending() → queue.takeMerged() → send()
163+
164+
Inbound: channelRead() → onInbound() → validate & deduplicate
165+
→ broadcastInbound() → queue.addPublish() → flushPending()
166+
```
167+
168+
**Gossip-Specific:**
169+
- **`GossipRouter`** extends `AbstractRouter` with mesh topology management
170+
- Heartbeat mechanism for GRAFT/PRUNE/IHAVE/IWANT control messages
171+
- Peer scoring for spam resistance
172+
- Control messages batched via `GossipRpcPartsQueue`
173+
174+
**Key Flush Triggers:**
175+
- After processing inbound messages (sync validation complete)
176+
- After async message validation completes
177+
- On peer activation (sends initial subscriptions)
178+
- During Gossip heartbeat (mesh management operations)
179+
- After explicit publish/subscribe API calls
180+
181+
### Multistream Protocol Negotiation
182+
183+
**`MultistreamProtocol`** (`protocol/multistream/MultistreamProtocol.kt`):
184+
- Used at three layers: security negotiation, muxer negotiation, protocol negotiation
185+
- Contains list of `ProtocolBinding`s with protocol names
186+
- Delegates to `Negotiator` (initiator/responder)
187+
- Completes with `ProtocolSelect<T>` containing selected protocol handler
188+
189+
**Pattern:** Any negotiable component extends `ProtocolBinding<T>`:
190+
- Security channels, stream muxers, application protocols all use this pattern
191+
192+
## Development Patterns
193+
194+
### Netty Integration
195+
196+
All protocol logic is implemented as Netty `ChannelHandler`s:
197+
- **`P2PChannelOverNetty`**: Base wrapper for both `Connection` and `Stream`
198+
- **`ConnectionOverNetty`**: Wraps connection-level channel with secure and muxer sessions
199+
- **`StreamOverNetty`**: Wraps stream-level channel with protocol negotiation
200+
201+
### Async Pattern
202+
203+
Extensive use of `CompletableFuture<T>` for async operations:
204+
- Protocol negotiation with timeouts
205+
- Connection establishment across multiple addresses
206+
- Message publishing and validation
207+
208+
### Event Thread Safety
209+
210+
The pub/sub system (and other components) use single-threaded event loops via `P2PService`:
211+
- All operations run on `executor: ScheduledExecutorService`
212+
- Components like `RpcPartsQueue` are explicitly "NOT thread safe" but guaranteed single-threaded access
213+
- Methods: `runOnEventThread {}`, `submitOnEventThread {}`, `submitAsyncOnEventThread {}`
214+
215+
### Testing Patterns
216+
217+
**JUnit 5** with:
218+
- `@Test` for standard tests
219+
- `@ParameterizedTest` with `@MethodSource` for data-driven tests
220+
- AssertJ for fluent assertions (`assertThat(...)`)
221+
- MockK for mocking
222+
223+
**Test Infrastructure:**
224+
- Test fixtures in `src/testFixtures/` for shared test utilities
225+
- Host builder DSL used extensively in tests
226+
- `TestChannel` and `TestLogAppender` utilities
227+
228+
**Example Test Pattern (from GossipRpcPartsQueueTest):**
229+
```kotlin
230+
@ParameterizedTest
231+
@MethodSource("testCases")
232+
fun `test message merging`(params: GossipParams, queue: TestQueue) {
233+
val monolith = queue.mergedSingle() // Ground truth
234+
val split = queue.takeMerged() // Actual implementation
235+
236+
// Verify limits respected
237+
assertThat(split).allMatch { router.validateMessageListLimits(it) }
238+
239+
// Verify semantic equivalence
240+
assertThat(split.merge().disperse()).isEqualTo(monolith.disperse())
241+
}
242+
```
243+
244+
### Code Style
245+
246+
- Kotlin 1.6 with JVM target 11
247+
- ktlint formatting (run `./gradlew spotlessApply`)
248+
- Detekt static analysis
249+
- Wildcard imports allowed
250+
- No trailing commas enforced
251+
- All warnings as errors (`allWarningsAsErrors = true`)
252+
253+
## Important Implementation Details
254+
255+
### Protobuf Code Generation
256+
257+
Protobuf definitions in `src/main/proto/` are compiled via `com.google.protobuf` Gradle plugin.
258+
Generated code in `build/generated/source/proto/main/java/`.
259+
260+
To regenerate: `./gradlew :libp2p:clean :libp2p:build`
261+
262+
### Multiaddr Format
263+
264+
Network addresses use multiaddr format:
265+
- Example: `/ip4/127.0.0.1/tcp/30333/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N`
266+
- Parsed/managed in `core/multiformats/`
267+
- Each transport validates specific multiaddr components
268+
269+
### PeerId Generation
270+
271+
`PeerId` is derived from peer's public key:
272+
- Multihash of the public key bytes
273+
- 32-50 bytes depending on key type
274+
- Supports RSA, Ed25519, Secp256k1, ECDSA
275+
276+
### Security Handshake Timeout
277+
278+
Default timeout for security handshakes: **5 seconds**
279+
- Applies to Noise and TLS handshakes
280+
- Configurable in protocol implementations
281+
282+
## Common Development Workflows
283+
284+
### Adding a New Protocol
285+
286+
1. Define protocol binding with multistream name (e.g., `/myapp/myprotocol/1.0.0`)
287+
2. Implement `ProtocolHandler<TController>` with initiator/responder logic
288+
3. Register with Host via `protocols { add(...) }` in builder
289+
4. Implement controller interface for protocol operations
290+
291+
See `examples/chatter/` for a complete example.
292+
293+
### Adding a New Transport
294+
295+
1. Extend `Transport` interface
296+
2. Implement `listen()` and `dial()` for raw connection establishment
297+
3. Delegate to `ConnectionUpgrader` for security/muxer negotiation
298+
4. Add multiaddr parsing logic for transport-specific components
299+
5. Register with Host via `transports { add(...) }`
300+
301+
### Debugging Connection Issues
302+
303+
- Use `ConnectionVisitor` and `StreamVisitor` for lifecycle observation
304+
- Enable debug logging for `io.libp2p` package
305+
- Check multiaddr format compatibility between peers
306+
- Verify protocol versions match (especially for security/muxer)
307+
308+
### Working with Pub/Sub
309+
310+
- All pub/sub operations run on event thread (thread-safe by design)
311+
- Message validation happens before broadcasting
312+
- Seen cache prevents duplicate message processing
313+
- Control messages automatically batched for efficiency
314+
- Gossip mesh heartbeat runs every 1 second (default)

0 commit comments

Comments
 (0)