Support explore resolution mode

[oed]

Based on this discussion ipfs/ipfs-webui#2363 it seems like the Kubo webui isn't the right place to implement a general explorer interface for IPFS that is usable by the average web user. The IPLD explorer was given as an alternative but is to technical for the average user.

However, it seems like inbrowser.tld supports showing directories if you resolve content that doesn't serve a web app (e.g. index.html). For example, this is the contents of dapprank.eth/dapps: https://bafybeibpfbdvt3kfuqap3sqbi5lqjcgdbmzjchxkiiqcmmnmdesglxt7wm.ipfs.inbrowser.link/

After a discussion with @aschmahmann he came up with the idea of adding a flag to disable resolving web content (e.g. index.html). I would imagine this could be implemented as something like ?explore=true. For example https://bafybeicqsmt6tunfphct5dczrnnf23yculkw3t3i2rkjfjtbo3eklmgfhu.ipfs.inbrowser.link?explore=true should show the folder content rather than the webapp.

[SgtPooki]

@oed this is interesting. I would be open to this one and will discuss it with teammates during the Helia WG today.

One thing about the query parameter: I've been namespacing all query parameters, prefixing them with ipfs-sw so that it's unlikely any query parameters intended to be consumed by the service-worker-gateway will interfere with query parameters different websites rendered by it would use.

[oed]

@SgtPooki cool. Another approach Adin suggested was to maybe make it a separate deployment, e.g. inbrowser-explorer.link (insert better domain name). So that a param isn't needed, just make it a core config in the deployment.

Not sure which is easier for you though.

[lidel]

I agree with @SgtPooki that adding query option for disabling "web" features (index.html, _redirects) feels useful, however this feels useful beyond service worker gateway.

This project has a specific purpose, aims to follow https://specs.ipfs.tech/http-gateways/ and deployment at https://inbrowser.link aims to be a drop-in replacement for subdomain gateway at dweb.link.
Adding such feature here and not everywhere else will be bad for users (we aim at feature parity).

We can prototype here, but should aim at IPIP against https://specs.ipfs.tech/http-gateways/ so the spec be also implemented in GO (boxo, kubo, rainbow). This way you would be able to use the same syntax everywhere (including ipfs-desktop's gateway).

@oed are web-specific features index.html and _redirects only things you want to disable?

I suspect there is more to this:

• If we frame this as format=web|no-web (web being implicit default), the no-web could not only disable index.html and _redirects, but also return web content types JS/CSS/HTML as Content-Type: text/plain, allowing for meaningful inspection right from the browser.
  • We have prior art that changes the web behavior via ?download.
  • We could also have a global flag in Kubo/Rainbow that disables support for format=web, using no-web by default, allowing gateway operators to run deserialized responses without the risk of being used in phishing campaigns.

If this sounds sensible, next step would be PR draft in https://github.com/ipfs/specs/ that explains how new parameter works, and then PR with implementation in GO and JS.

ps. https://explore.ipld.io/#/explore/bafybeiahtlgk2xhjl7eveyfplbsqo6ztdb72wonlrednz5sk4x3ieofy6m mentioned in ipfs/ipfs-webui#2363 works fine. It uses the same mechanism as this project for retrieval.

[oed]

Thanks for this overview @lidel. That all makes sense. Will see when I can follow up with a IPIP.

If we frame this as format=web|no-web (web being implicit default), the no-web could not only disable index.html and _redirects, but also return web content types JS/CSS/HTML as Content-Type: text/plain, allowing for meaningful inspection right from the browser.

For me it would be ideal if it looks more like the file explorer on the webui, e.g. preview files, not downloading them. Imo this is the biggest missing piece of the IPFS ecosystem, something that serves a similar purpose to Etherscan on Ethereum. Understand that this would be a bigger undertaking though.

ps. https://explore.ipld.io/#/explore/bafybeiahtlgk2xhjl7eveyfplbsqo6ztdb72wonlrednz5sk4x3ieofy6m mentioned in ipfs/ipfs-webui#2363 works fine. It uses the same mechanism as this project for retrieval.

Is it using the same mechanism? For some reason I'm more frequently getting errors from trying to resolve content from there.

[lidel]

When you frame it this way, I really think you describe what https://explore.ipld.io/ aims to be: lower level inspector for IPFS ecosystem, for more technical people. Perhaps more cost-effective way is to improve https://github.com/ipld/explore.ipld.io/ instead? (Improving single web app is easier than making changes to gateway protocol across the stack and both GO and JS).

@SgtPooki may confirm, but afaik if you dont set up any custom CORS in your local Kubo/IPFS Desktop, the https://explore.ipld.io/ should use the same in-browser Helia backend as https://inbrowser.link, and virtually have the same success rate.

If they differ, it is possible https://github.com/ipld/explore.ipld.io/ uses older version of Helia and needs to be updated.

[oed]

When you frame it this way, I really think you describe what https://explore.ipld.io/ aims to be: lower level inspector for IPFS ecosystem, for more technical people.

Hm, this isn't really what I'm saying though. A lot of people use Etherscan as a way to explore what's going on onchain. Not only technical ppl. What I'm looking for is really something similar for IPFS, i.e. a way to explore files and folders hosted on IPFS in a user friendly way.

If they differ, it is possible https://github.com/ipld/explore.ipld.io/ uses older version of Helia and needs to be updated.

This is quite likely imo. I get like a 50% success rate on explore.ipld.io while on inbrowser.link I get basically 100%.

[SgtPooki]

@oed success rate issues on explore.ipld.io sound like a bug with that repo.. I can look into that

[SgtPooki]

Thoughts we need to discuss later:

Do we envision a world where a flag like ?explore=json,?explore=html, or ?explore (default to json) could yield a different output from verified-fetch? I imagine some normalized obj representation of content (similar to that of the internals of ipld-explorer-components) could be useful, and then we could iterate on it from there.

I feel like it would make sense to get some specs written up for a JSON/html preview of content similar to a blockchain transaction viewer.

I have been trying to keep this repo's issues and PRs tidy and this issue keeps popping up and i'm not quite sure if we want to just punt this over to verified-fetch or the specs repo