feat: generates parts of the dashboard async, using ipfs/gateway-conformance

[laurentsenta]

The ipfs/gateway-conformance has reached a point where we should be able to generate most of the info needed by our public gateway checker. Reusing the project should reduce maintenance and avoid the "Red Message Of Death" caused by the user's browser connecting with many gateways.

Interop

• Agree on the output format we want to share between conformance and the frontend

conformance.json:

{
  "gateway.ipfs.io": {
    "metadata": { "timestamp": ...., "country": .... },
    "results": {
          "CORS": { "success": 12, "fail": 0, "skipped": 0 },
          "IPNS": ...,
    }
   },
  "another-gateway.link": ...
}

CI: Call the gateway conformance workflow - #450

@laurentsenta

• Reuse the workflow from e2e production workflow
• Make the list of gateways that live in gateways.ts available to the workflow
• create a minimal output we can render in the CI
• Extract the results in a format that can be reused here
  • Export TestGroups metadata
  • Aggregates results with a piece of js

CI: Generate metadata (country) on the server side - #451

Frontend: Render the results

• import jsons to render static results
  • conformance.json
• "Online" column - Add opt-in "online" + latency check on the client side #452

Make the tests pass

• likely we'll have issues with missing fixtures

Follow-ups

• Create issues from notes in feat: generates parts of the dashboard async, using ipfs/gateway-conformance #442 (comment)
• Close duplicate tasks from feat: generates parts of the dashboard async, using ipfs/gateway-conformance #442 (comment)

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[laurentsenta]

(self-assigning, I'll take the responsibility of moving this work forward, but happy to collaborate)

[whizzzkid]

Thanks for submitting this story @laurentsenta, excellent effort, we can pair on this if you'd like!

Can we mark these issues as duplicate/subtask of this issue (as I feel these can have a potential overlap):

• Refactor GatewayNode Checks #311
• feature enhancement: Automate removal of invalid gateways #281
• Github actions should test new gateway PRs #248
• Abstract and encapsulate gateway checking logic from this package and JayBrown/Tools/tree/master/ipfg #192

CC: @SgtPooki

[laurentsenta]

@SgtPooki @whizzzkid I updated the plan and prepared the PR for the CI side.

@lidel currently the gateway check looks like this:

the conformance test suite granularity is much finer:

(from CI)

Proposal: we define TestGroups metadata in conformance testing that will be used in project like this one.

Related PR: ipfs/gateway-conformance#130