Research ways we could leverage /webtransport peers

[lidel]

This is a placeholder for work that will improve connectivity
by leveraging peers that expose HTTP Gateway over WebTransport with certhash.

Not ready yet, but will update this issue as relevant building blocks land in ecosystem.

High level idea

• libp2p supports webtransport (https://github.com/libp2p/specs/tree/master/webtransport)
  • why it matters? it allows self-signed certificates in web browser without triggering mixed-content warning, allows for end-to-end encryption without having to get TLS cert from a CA (makes IPFS node deployment way way easier)
  • webtransport listener is enabled by default in Kubo, which is ~80% of IPFS nodes
• libp2p wants to expose HTTP support over arbitrary transports (incl. webtransport)
• just like we reuse HTTP client from Chromium to fetch blocks over plain HTTP(S), we could also ook into reusing WebTransport implementation to fetch blocks [over HTTP over [lipbp2p connection [over WebTransport]]]
• when ipfs-chrome is unable to fetch CID from a gateway, it could do lookup over /routing/v1 endpoint (https://specs.ipfs.tech/routing/http-routing-v1/, https://github.com/protocol/bifrost-infra/issues/2142) to learn about addresses of peers that speak /webtransport

Open questions

• is this worth it?
  • not today, but when we have https://github.com/protocol/bifrost-infra/issues/2142 and add HTTP spec libp2p/specs#508 and it is used for exposing trustless gateway semantics over libp2p /webtransport connection it will grow the pie of possible sources of blocks
• how the client can tell a peer speaks http gateway over /webtransport?
  • Document /http libp2p/specs#550 and add HTTP spec libp2p/specs#508
    • TBD: use this for exposing /ipfs/{cid} block/car in Kubo
  • IPIP-425: Signaling Features on HTTP Gateways ipfs/specs#425
    • TBD: use this in boxo/gateway
• what would be the absolute minimum amount of libp2p we would have to reimplement here?

[lidel]

@MarcoPolo what would be the absolute minimum amount of libp2p we would have to implement here to:

1. take /..../webtransport/../p2p/{peerid} multiaddr
2. connect to it
3. check if it supports /http* and probe for /ipfs endpoint.
4. make HTTP rrquests over /webtransport connection that exposes /ipfs` endpoints semantics
   ?

[MarcoPolo]

At a very high level:

1. take /..../webtransport/../p2p/{peerid} multiaddr

a. Parse the multiaddr.
b. Parse multihash (certhashes).
c. Parse the peer id.

2. connect to it

Right now this requires peer authentication. There's interest to make WebTransport authentication optional in the future, which would simplify this use case.

a. Open a WebTransport connection to https://example.com/.well-known/libp2p-webtransport?type=noise
b. Due to peer auth, run a libp2p noise handshake: https://github.com/libp2p/specs/tree/master/noise#the-noise-handshake
c. If doing peer auth, verify the certhashes of the peer are what you expect.

3. check if it supports /http* and probe for /ipfs endpoint.

a. Open a new stream and negotiate /http/1.1 on it. If this worked they support HTTP over libp2p.
b. GET the .well-known/libp2p resource and see where the trustless-gateway is mounted.

4. Get blocks!

a. Open a new stream with /http/1.1 for each HTTP request.
b. Use js-libp2p-fetch instead of fetch to get the blocks. (js-libp2p-fetch doesn't actually need js-libp2p, and is almost 0 dependencies except for a js-libp2p quirk requiring a Uint8ArrayList dependency).
c. At this point it should be identical to the existing way.

If we prioritize this we could make webtransport auth optional, and thus reduce a lot of complexity around doing the Noise handshake.

[lidel]

Update: Kubo 0.23 will include opt-in "gateway over libp2p" experiment from ipfs/kubo#10108

TLDR

• it will expose the same HTTP semantics as https://specs.ipfs.tech/http-gateways/trustless-gateway/, but over Libp2p stream, instead of TCP HTTP stream
  • Only caveat is that this gateway is not recursive, it won't return HTTP 200 for CID it does not have in local repository. This is useful only when we've learned the peer is a provider for CID we look for.
  • ipfs-chromium could use this as a fallback when no gateway has the data we look for
    • 1. query https://specs.ipfs.tech/routing/http-routing-v1/
    • 2. find peers that speak /webtransport (or /https or /http/tls -- these are plain HTTP gateways)
    • 3. try fetching from them
• this allows making the same HTTP GET requests for Blocks and CARs, but over libp2p socket, and the transport could be over WebTransport, which does not require TLS cert from CA, as long /certhash is included in the multiaddr (e.g. /ip4/{ip}/udp/{port}/quic-v1/webtransport/certhash/{chash}

I think it is long term thing, but we should be able to make it work:

• /routing/v1 most likely will be enabled by default on gateways in the future, allowing for lookup for additional gateways/providers
  • /https and /http/tls could work today – I've filled separate Leverage /https peers #66 with details
  • /webtransport will require more effort, implementing a bare minimum of libp2p necessary for establishing connection to get /http/1.1 socket.
    • To ensure user privacy we could be generating random PeerID per peer + rotated every time browser starts.