bitswap: Split block responses into batches under 2 MiB #516

dmitry-markin · 2026-01-16T18:07:27Z

Split blocks in a Bitswap response into batches under 2 MiB so the maximum substream message size is respected and we don't lose the blocks.

Closes #514.

Follow-ups:

Add tests
Release litep2p with the fix & upgrade litep2p in polkadot-sdk

src/protocol/libp2p/bitswap/config.rs

lexnv · 2026-01-19T10:59:23Z

src/protocol/libp2p/bitswap/mod.rs

+        if message.len() <= config::MAX_MESSAGE_SIZE {
+            tracing::trace!(
+                target: LOG_TARGET,
+                cid_count,
+                "sending Bitswap presence message",
+            );
+            match tokio::time::timeout(WRITE_TIMEOUT, substream.send_framed(message)).await {
+                Err(_) => return Err(Error::Timeout),
+                Ok(Err(e)) => return Err(Error::SubstreamError(e)),
+                Ok(Ok(())) => {}
+            }
+        } else {
+            // This should never happen in practice, but log a warning if the presence message
+            // exceeded [`config::MAX_MESSAGE_SIZE`].
+            tracing::warn!(
+                target: LOG_TARGET,
+                size = message.len(),
+                max_size = config::MAX_MESSAGE_SIZE,
+                "outgoing Bitswap presence message exceeded max size",
+            );


nit: this is similar to the block below, maybe we can group them to avoid duplications?

I can't think of a good way of de-duplicating this, the data and the log messages are different.

well, I agree with Alex, this could be maybe de-duplicated. In both cases, you are building let message = schema::bitswap::Message, so when building it we could check MAX_MESSAGE_SIZE or MAX_BATCH_SIZE.

Basically, we could reuse extract_next_batch(&mut blocks, config::MAX_BATCH_SIZE) for both - maybe with closure which would return length (for presence - 2 bytes?, for block real length)

lexnv · 2026-01-19T11:02:17Z

src/protocol/libp2p/bitswap/mod.rs

+                    "sending Bitswap blocks message",
+                );
+                match tokio::time::timeout(WRITE_TIMEOUT, substream.send_framed(message)).await {
+                    Err(_) => return Err(Error::Timeout),


dq: This will drop any remaining block because we consider the connection as unhealthy?

However, this might be expected under heavy load that the protocol handle will not be able to keep up with messages. Maybe we can propagate this to higher levels or is the timeout error sufficient to retry later on?

This is a good question I don't have the right an answer to. Indeed the entire response will be dropped. I am inclined to think that if the protocol handle is not able to keep up with messages, we shouldn't retry sending. Instead it will be up to the Bitswap client code to handle the timeout and repeat the query, may be querying another peer.

It would be good to investigate how Kubo handles this and if this will automatically work.

src/protocol/libp2p/bitswap/mod.rs

lexnv

Nice one! 👍

src/protocol/libp2p/bitswap/config.rs

src/protocol/libp2p/bitswap/mod.rs

bkontur · 2026-01-20T11:48:51Z

src/protocol/libp2p/bitswap/mod.rs

+
+    let count = message.payload.len();
+
+    (count > 0).then(|| (message.encode_to_vec().into(), count))


How does Bitswap work, shouldn't we send also empty Message? Doesn't it cause disconnect or something?

If we passed an empty iterator (e.g., no block presence at all), it doesn't make sense to send empty message.

bkontur · 2026-01-20T11:51:45Z

src/protocol/libp2p/bitswap/mod.rs

+
+    while let Some(batch) = extract_next_batch(&mut blocks, config::MAX_BATCH_SIZE) {
+        if let Some((message, block_count)) = blocks_message(batch) {
+            if message.len() <= config::MAX_MESSAGE_SIZE {


Basically, this IF does not make sense, if we ensure that MAX_BATCH_SIZE < MAX_MESSAGE_SIZE

It is highly unlikely in practice, but due to protobuf overhead message size can be > MAX_BATCH_SIZE.

src/protocol/libp2p/bitswap/mod.rs

bkontur · 2026-01-20T11:54:17Z

src/protocol/libp2p/bitswap/config.rs


-/// Maximum Size for `/ipfs/bitswap/1.2.0` substream payload. Note this is bigger than 2 MiB max
-/// block size to account for protobuf message overhead.
-const MAX_PAYLOAD_SIZE: usize = 2_100_000;


@rosarp probably this constant was the reason, why 2 MiB didn't work :)

In the past it was exactly 2 MiB, but due to prefix and protobuf overhead the message didn't fit.

bkontur · 2026-01-20T11:56:33Z

src/protocol/libp2p/bitswap/config.rs

+
+/// Maximum batch size of all blocks in a single Bitswap message combined. Enforced on the
+/// application protocol level.
+pub const MAX_BATCH_SIZE: usize = 2 * 1024 * 1024;


I am just thinking, what about adding both to the Config:

pub struct Config { /// Protocol name. pub(crate) protocol: ProtocolName, /// Protocol codec. pub(crate) codec: ProtocolCodec, /// TX channel for sending events to the user protocol. pub(super) event_tx: Sender<BitswapEvent>, /// RX channel for receiving commands from the user. pub(super) cmd_rx: Receiver<BitswapCommand>, pub max_message_size: usize, pub max_batch_size: usize, }

And use those constants for default value?

2 MiB are defined in the protocol spec, so i don't think we should add the possibility to "tune" it.

bkontur · 2026-01-20T12:01:33Z

src/protocol/libp2p/bitswap/mod.rs

+    let mut block_count = 0;
+
+    for b in blocks.iter() {
+        let next_block_size = b.1.len();


Does this batching also count with prefix and struct bytes overhead (2-3 bytes)?

schema::bitswap::Block { prefix, data: block, });

No, it's only about total block size. As per spec, 2 MiB blocks must go through, so the actual message size will be higher.

bkontur

Looks good, let's release and bump master.
Left just couple of ultra-nits, which could be also ignored, maybe just rename max_size -> max_batch_size for clarity

Co-authored-by: Branislav Kontur <[email protected]>

bkontur

@dmitry-markin cool, thank you, let's ship it :)

dmitry-markin · 2026-01-20T13:15:44Z

We'll create a release once another bugfix is merged (#518). Hopefully today.

## [0.13.0] - 2026-01-21 This release brings multiple fixes to both the transport and application-level protocols. Specifically, it enhances WebSocket stability by resolving AsyncWrite errors and ensuring that partial writes during the negotiation phase no longer trigger connection failures. At the same time, Bitswap client functionality is introduced, which makes this release semver breaking. ### Added - Add Bitswap client ([#501](#501)) ### Fixed - notif/fix: Avoid CPU busy loops on litep2p full shutdown ([#521](#521)) - protocol: Ensure transport manager knows about closed connections ([#515](#515)) - substream: Decrement the bytes counter to avoid excessive flushing ([#511](#511)) - crypto/noise: Improve stability of websockets by fixing AsyncWrite implementation ([#518](#518)) - bitswap: Split block responses into batches under 2 MiB ([#516](#516)) - crypto/noise: Fix connection negotiation logic on partial writes ([#519](#519)) - substream/fix: Fix partial reads for ProtocolCodec::Identity ([#512](#512)) - webrtc: Avoid panics returning error instead ([#509](#509)) - bitswap: e2e test & max payload fix ([#508](#508)) - tcp: Exit connections when events fail to propagate to protocols ([#506](#506)) - webrtc: Avoid future being dropped when channel is full ([#483](#483)) --------- Co-authored-by: Alexandru Vasile <[email protected]>

bitswap: Split block responses into batches under 2 MiB

4b6a21c

dmitry-markin requested a review from lexnv January 16, 2026 18:07

dmitry-markin self-assigned this Jan 16, 2026

dmitry-markin added this to Networking Jan 16, 2026

dmitry-markin mentioned this pull request Jan 16, 2026

[Bitswap] Handle big data #514

Closed

dmitry-markin added 3 commits January 16, 2026 20:15

minor: make clippy happy

a287a3f

Extract chunking into testable func

4a30464

Add tests

2fa321c

dmitry-markin requested a review from bkontur January 19, 2026 09:14

Merge remote-tracking branch 'origin/master' into dm-bitswap-batch

6aa634e

dmitry-markin marked this pull request as ready for review January 19, 2026 09:15

lexnv reviewed Jan 19, 2026

View reviewed changes

src/protocol/libp2p/bitswap/config.rs Outdated Show resolved Hide resolved

lexnv reviewed Jan 19, 2026

View reviewed changes

src/protocol/libp2p/bitswap/mod.rs Show resolved Hide resolved

lexnv approved these changes Jan 19, 2026

View reviewed changes

bkontur reviewed Jan 20, 2026

View reviewed changes

src/protocol/libp2p/bitswap/config.rs Outdated Show resolved Hide resolved

Apply review suggestions

49f7fdc

bkontur reviewed Jan 20, 2026

View reviewed changes

src/protocol/libp2p/bitswap/mod.rs Outdated Show resolved Hide resolved

bkontur reviewed Jan 20, 2026

View reviewed changes

src/protocol/libp2p/bitswap/mod.rs Outdated Show resolved Hide resolved

bkontur reviewed Jan 20, 2026

View reviewed changes

bkontur approved these changes Jan 20, 2026

View reviewed changes

dmitry-markin and others added 3 commits January 20, 2026 14:45

Update src/protocol/libp2p/bitswap/mod.rs

811d82a

Co-authored-by: Branislav Kontur <[email protected]>

Apply review suggestions - 2

2527ad9

Merge remote-tracking branch 'origin/master' into dm-bitswap-batch

b1e01f5

bkontur approved these changes Jan 20, 2026

View reviewed changes

dmitry-markin merged commit 0d2d1bd into master Jan 20, 2026
8 checks passed

dmitry-markin deleted the dm-bitswap-batch branch January 20, 2026 13:12

github-project-automation bot moved this to Blocked ⛔️ in Networking Jan 20, 2026

dmitry-markin mentioned this pull request Jan 20, 2026

chore: Release litep2p v0.13.0 #522

Merged


		let count = message.payload.len();

		(count > 0).then(\|\| (message.encode_to_vec().into(), count))

bitswap: Split block responses into batches under 2 MiB #516

bitswap: Split block responses into batches under 2 MiB #516

Uh oh!

Conversation

dmitry-markin commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lexnv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bkontur left a comment

Choose a reason for hiding this comment

Uh oh!

bkontur left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dmitry-markin commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dmitry-markin commented Jan 16, 2026 •

edited

Loading