Skip to content

feat: add ipns reproviding #764

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

feat: add ipns reproviding #764

wants to merge 5 commits into from

Conversation

2color
Copy link
Member

@2color 2color commented Mar 13, 2025

What

This PR adds the functionality necessary for a Helia node to reprovide records it created before expiry.

Why

Currently, the ipns.republish method which runs the republishing loop does not actually handle republishing.

How

  • Add the the ability to iterate on records in the data store.

Open question about storing keys

  • We don't store the private key when publish is called. This means that we can only reprovide valid IPNS records, but with the current defaults (48 hour lifetime and 23 hour republish interval), that means one round of republishing, after which all stored IPNS records become invalid.
    -This begs the question of whether we should store the keys so that we can create new valid records for stored IPNS names.
  • After looking at how we manage the self key for the libp2p peer we create, it seems like we probably want to add the keychain. The question then is whether to add the keychain to the Helia interface or just the IPNS module.
  • Adding a keychain would allow us to iterate on keys rather than IPNS names when republishing. Though I'm not sure if there's any obvious benefit to this.

publish(key: PrivateKey, value: CID | PublicKey | MultihashDigest<0x00 | 0x12> | string, options?: PublishOptions): Promise<IPNSRecord>

2color and others added 5 commits February 24, 2025 13:45
* origin/main:
  chore: use peer id parsing function from libp2p (#762)
  feat: add republish signed ipns records (#745)
  fix: use bytestream methods to add byte streams (#758)
  chore: bump codecov/codecov-action from 5.3.1 to 5.4.0 (#752)
  feat: allow modifying trustless-gateway fetch (#751)
  fix: align implicit default ttl with specs (#749)
  docs: add spell checker to ci (#743)
  chore: Update FUNDING.json for Optimism RPF (#746)
@2color 2color changed the title feat: add ipns republishing feat: add ipns reproviding Mar 26, 2025
@2color 2color added kind/enhancement A net-new feature or improvement to an existing feature status/blocked Unable to be worked further until needs are met labels Apr 4, 2025
@2color
Copy link
Member Author

2color commented Apr 4, 2025

I'll wait for feedback from @achingbrain regarding key storage and the necessity of the keychain (see PR description).

this.log(`found ${recordsToRepublish.length} records to republish`)

// Republish each record
for (const { routingKey, record } of recordsToRepublish) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to do this in a queue so we can publish multiple records in parallel.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The queue implementation we use elsewhere in the stack is this one, it has simple concurrency controls, etc.

@2color
Copy link
Member Author

2color commented Apr 30, 2025

Following a discussion offline with @achingbrain, we need to move key management internally to implement reproviding.

Key management is typically (and store encrypted at rest) handled by the js-libp2p keychain, however we don't have libp2p in @helia/http (which can also be used for @helia/ipns with delegated routers). This means that we probably need a dedicated keychain instance in the IPNS class, so that we don't depend on the libp2p instance.

@achingbrain
Copy link
Member

achingbrain commented Apr 30, 2025

This means that we probably need a dedicated keychain instance in the IPNS class

@helia/http could also use a libp2p instance that's only configured with HTTP routers? Then you don't need to handle things differently at this level.

It'd also be good to see just how lightweight we can make libp2p for this sort of use-case.

@2color
Copy link
Member Author

2color commented Apr 30, 2025

@helia/http could also use a libp2p instance that's only configured with HTTP routers? Then you don't need to handle things differently at this level.

Yeah we could try that first and see how it feels. It may increase the bundle size.

I also vaguely remember there being a bug with libp2p whereby instantiating it without the DHT, causes an infinite random walk loop with the defaults, as it's looking for a circuit relay reservation by default.

Would this also allow Helia to make use of the libp2p over http stuff you've be working on?

@achingbrain
Copy link
Member

It may increase the bundle size.

Looking at the breakdown, the usual suspects are already there - @libp2p/crypto, @noble/hashes, etc so it might not increase it by that much.

I also vaguely remember there being a bug with libp2p whereby instantiating it without the DHT, causes an infinite random walk loop with the defaults, as it's looking for a circuit relay reservation by default.

I'd imagine this configuration wouldn't configure a relay listener so it wouldn't try to find a relay?

We need this landing, really - ipfs/specs#476

Would this also allow Helia to make use of the libp2p over http stuff you've be working on?

Yes, though you'd need transports etc in order to use libp2p for the transport layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature status/blocked Unable to be worked further until needs are met
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants