Skip to content

Reduce embedded IPFS node resource usage - #2

Merged
aopoltorzhicky merged 2 commits into
masterfrom
refactoring/ipfs-node
Jul 10, 2026
Merged

Reduce embedded IPFS node resource usage#2
aopoltorzhicky merged 2 commits into
masterfrom
refactoring/ipfs-node

Conversation

@aopoltorzhicky

Copy link
Copy Markdown
Member

Profiling a metadata-indexer instance showed that ~95% of memory (~2GB RSS) and CPU
was consumed by the embedded IPFS node acting as a full network participant:
announcing every cached block to the DHT and serving blocks to other peers.

Changes

  • Apply config to existing repositories. fsrepo.Init is a no-op when the repo
    already exists, so config changes in code never reached deployed nodes. The config
    is now persisted on every start via repo.SetConfig (identity is preserved).
  • Disable providing and bitswap server (Provide.Enabled=false,
    Bitswap.ServerEnabled=false) — the node is a pure consumer.
  • Switch to delegated routing (Routing.Type=delegated) — provider lookups go
    through HTTP routers instead of running a DHT client.
  • Lower connection manager watermarks to 20/50 with a 1m grace period.
  • Bound the datastore: StorageMax=2GB + a periodic GC goroutine
    (corerepo.PeriodicGC); the enable-gc ExtraOpt is daemon-only and had no effect.
  • Skip repo re-initialization on start when it already exists (avoids generating
    and discarding an RSA key on every launch).
  • Remove the reconnect goroutine that logged the full peer list every 3 minutes.

Tests

Added tests covering repo creation (idempotency, identity preservation) and config
persistence/override on existing repos — no node startup or network required.

@aopoltorzhicky
aopoltorzhicky merged commit a8e0bfa into master Jul 10, 2026
5 checks passed
@aopoltorzhicky
aopoltorzhicky deleted the refactoring/ipfs-node branch July 10, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant