Skip to content

feat(config): expose ProviderSearchMaxResults #10773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lidel merged 1 commit into from
Apr 9, 2025
+22 −12
Merged

feat(config): expose ProviderSearchMaxResults #10773

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/internal.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ type InternalBitswap struct {
EngineTaskWorkerCount OptionalInteger
MaxOutstandingBytesPerPeer OptionalInteger
ProviderSearchDelay OptionalDuration
ProviderSearchMaxResults OptionalInteger
WantHaveReplaceSize OptionalInteger
}
9 changes: 6 additions & 3 deletions core/node/bitswap.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ const (
DefaultEngineTaskWorkerCount = 8
DefaultMaxOutstandingBytesPerPeer = 1 << 20
DefaultProviderSearchDelay = 1000 * time.Millisecond
DefaultMaxProviders = 10 // matching BitswapClientDefaultMaxProviders from https://github.com/ipfs/boxo/blob/v0.29.1/bitswap/internal/defaults/defaults.go#L15
Copy link
Copy Markdown
Member Author

@lidel lidel Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open question: should we keep 10, or switch to 0 (unlimited, like in rainbow?)

Copy link
Copy Markdown
Contributor

@guillaumemichel guillaumemichel Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC MaxProviders in the context of bitswap is the max number of providers that Bitswap asks the content routing system (Amino DHT) to return, if more are returned only the first MaxProviders are read by Bitswap.

While it wouldn't be more effort for the content routing system to provide more providers (as long as they exist), we may want to bound the number of providers records bitswap/http will try to fetch the content from, for a single CID.

Also, with a MaxProviders that is too low, we may not be able to discover the content, if was advertised by many unstable nodes that have gone offline since.

Increasing DefaultMaxProviders to something like 32 could be a middle ground in ensuring content is likely to be found if there are online providers, while bounding retrieval attempts.

DefaultWantHaveReplaceSize = 1024
)

Expand Down Expand Up @@ -79,11 +80,13 @@ func Bitswap(provide bool) interface{} {

var provider routing.ContentDiscovery
if provide {
// We need to hardcode the default because it is an
// internal setting in boxo.
var maxProviders int = DefaultMaxProviders
if in.Cfg.Internal.Bitswap != nil {
maxProviders = int(in.Cfg.Internal.Bitswap.ProviderSearchMaxResults.WithDefault(DefaultMaxProviders))
}
pqm, err := rpqm.New(bitswapNetwork,
in.Rt,
rpqm.WithMaxProviders(10),
rpqm.WithMaxProviders(maxProviders),
rpqm.WithIgnoreProviders(in.Cfg.Routing.IgnoreProviders...),
)
if err != nil {
Expand Down
10 changes: 4 additions & 6 deletions docs/changelogs/v0.35.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This release was brought to you by the [Shipyard](http://ipshipyard.com/) team.
- [Overview](#overview)
- [🔦 Highlights](#-highlights)
- [Dedicated `Reprovider.Strategy` for MFS](#dedicated-reproviderstrategy-for-mfs)
- [`Routing.IgnoreProviders`](#routingignoreproviders)
- [Additional new configuration options](#additional-new-configuration-options)
- [Grid view in WebUI](#grid-view-in-webui)
- [📦️ Important dependency updates](#-important-dependency-updates)
- [📝 Changelog](#-changelog)
Expand All @@ -31,12 +31,10 @@ Users relying on the `pinned` strategy can switch to `pinned+mfs` and use MFS al

See [`Reprovider.Strategy`](https://github.com/ipfs/kubo/blob/master/docs/config.md#reproviderstrategy) for more details.

#### `Routing.IgnoreProviders`
#### Additional new configuration options

This new option allows ignoring specific peer IDs when returned by the content
routing system as providers of content. See the
[documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingignoreproviders)
for for information.
- [`Internal.Bitswap.ProviderSearchMaxResults`](https://github.com/ipfs/kubo/blob/master/docs/config.md##internalbitswapprovidersearchmaxresults) for adjusting the maximum number of providers bitswap client should aim at before it stops searching for new ones.
- [`Routing.IgnoreProviders`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingignoreproviders) allows ignoring specific peer IDs when returned by the content routing system as providers of content.

#### Grid view in WebUI

Expand Down
14 changes: 11 additions & 3 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ config file at runtime.
- [`Internal.Bitswap.EngineBlockstoreWorkerCount`](#internalbitswapengineblockstoreworkercount)
- [`Internal.Bitswap.EngineTaskWorkerCount`](#internalbitswapenginetaskworkercount)
- [`Internal.Bitswap.MaxOutstandingBytesPerPeer`](#internalbitswapmaxoutstandingbytesperpeer)
- [`Internal.Bitswap.ProviderSearchDelay`](#internalbitswapprovidersearchdelay)
- [`Internal.Bitswap.ProviderSearchDelay`](#internalbitswapprovidersearchdelay)
- [`Internal.Bitswap.ProviderSearchMaxResults`](#internalbitswapprovidersearchmaxresults)
- [`Internal.UnixFSShardingSizeThreshold`](#internalunixfsshardingsizethreshold)
- [`Ipns`](#ipns)
- [`Ipns.RepublishPeriod`](#ipnsrepublishperiod)
Expand Down Expand Up @@ -119,7 +120,7 @@ config file at runtime.
- [`Routing.Type`](#routingtype)
- [`Routing.AcceleratedDHTClient`](#routingaccelerateddhtclient)
- [`Routing.LoopbackAddressesOnLanDHT`](#routingloopbackaddressesonlandht)
- [`Routing.IgnoreProviders`](#routingignoreproviders)
- [`Routing.IgnoreProviders`](#routingignoreproviders)
- [`Routing.Routers`](#routingrouters)
- [`Routing.Routers: Type`](#routingrouters-type)
- [`Routing.Routers: Parameters`](#routingrouters-parameters)
Expand Down Expand Up @@ -1181,14 +1182,21 @@ deteriorate the quality provided to less aggressively-wanting peers.

Type: `optionalInteger` (byte count, `null` means default which is 1MB)

### `Internal.Bitswap.ProviderSearchDelay`
#### `Internal.Bitswap.ProviderSearchDelay`

This parameter determines how long to wait before looking for providers outside of bitswap.
Other routing systems like the Amino DHT are able to provide results in less than a second, so lowering
this number will allow faster peers lookups in some cases.

Type: `optionalDuration` (`null` means default which is 1s)

#### `Internal.Bitswap.ProviderSearchMaxResults`

Maximum number of providers bitswap client should aim at before it stops searching for new ones.
Setting to 0 means unlimited.

Type: `optionalInteger` (`null` means default which is 10)

### `Internal.UnixFSShardingSizeThreshold`

The sharding threshold used internally to decide whether a UnixFS directory should be sharded or not.
Expand Down
Loading