[Investigate/Research] "IPFS-lite" "light client" solutions (a.k.a. no Kubo needed)

[bkontur]

Related to:
#33
#90

Investigate how we could support an IPFS HTTP gateway for dApps without relying on external IPFS gateways.

The main idea is that Bulletin nodes already support IPFS over Bitswap (p2p). With some form of IPFS DHT auto-discovery (see #33 and an "intermediate" HTTP/p2p layer, we could implement everything directly within the dApps — using only a light client for Bulletin (or another chain) — and avoid connecting to external IPFS gateways.

Disclaimer: (Hope, I understood this idea well, based on discussion with @BigTava, which he had with Basti)

[dmitry-markin]

Can you explain what do you mean by "not relying on external IPFS gateways"? Do you mean the dApp should act as IPFS client?

I am a bit out of context and don't know why we need the HTTP gateway in the first place.

[bkontur]

HTTP gateway in the first place.

For example, only to to serve multi chunked content by one root CID or basically interpret IPFS DAG features, but I think with this: paritytech/polkadot-sdk#10301 we can support anything. Is this what you meant by #85 (comment) ?

[dmitry-markin]

HTTP gateway in the first place.

For example, only to to serve multi chunked content by one root CID or basically interpret IPFS DAG features, but I think with this: paritytech/polkadot-sdk#10301 we can support anything. Is this what you meant by #85 (comment) ?

Yes, if we handle DAG in the IPFS-native way, we won't need a separate gateway to assemble the chunks and can use standard IPFS API for this.

[dmitry-markin]

My understanding was that dApps should talk standard IPFS and don't need any Web2 gateways. At least once we have the DHT discovery layer working.

[karolk91]

I have prepared example apps here: #135. These use Typescript Helia library to connect via P2P (websocket) directly from the browser (web app) or terminal (cli app) to the Bulletin node (no Kubo software used)

Note: For the reason I haven't dig much into, I had to update the polkadot-sdk dependency, otherwise it was just timeouting when trying to fetch something by CID (#136) -- I assumed this is because of litep2p update i.e. it doesn't work on 0.11.0 but works fine on 0.12.0 and 0.12.2

Note2: I have also roughly tested #100 to check if other than default (raw) codec currently assumed would be also supported and dag-pb seemed to work just fine

[BigTava]

Is this still ongoing?

[bkontur]

Is this still ongoing?

@BigTava I left a comment in the issue you linked, basically @dmitry-markin is working on smoldot part (Web3 aligned), so we won't need helia or IPFS gateways (but still we could support both and can be also use - as Web2 and not Web3 aligned).