Discuss hardware listing changes

[MarceloRobert]

The hardware listing view is very slow, sometimes hitting timeouts. We need to define the proper approach to fix this problem.

• Should we go into the same direction as tree listing and make a table specifically for the hardware view and then update the values at insertion time (in the ingester)?
• Can we use materialized view(s)? when would it be updated? How would the query/queries be done?
• Is it possible to change how the hardware listing is sent to the frontend? Could we do like in tree listing, where we have an endpoint for the list itself, and another for the build/boot/test counts?
• We could also try a quick-ish solution first, and a more resilient solution later

Discussion first. Register in the comments any updates, so that we also have the datetime of the ideas

[tales-aparecida]

I think we need to check how the discussions about

Protocol proposal: https://docs.google.com/document/d/1Nbz79wrgIwKTadhIC543KkutHYsT64V0loaO9nYL8vk/edit?usp=sharing

Hardware registry schema proposal
https://lore.kernel.org/kernelci/2c821447e222e825b36bcc19917e6527d8358e8f.camel@pengutronix.de/T/#mfcb82ad79af84ff7c659c0374dccc9e5dd12a1c2

will reflect on the HW-list and future HW-detail

[MarceloRobert]

@tales-aparecida for the protocol proposal, I made a quick diagram of the use-cases similar to this comment:

---
config:
  layout: dagre
---
flowchart TD
 subgraph s1["Azure"]
        n2["KCIDB-Rest"]
        n5["Dashboard-Ingester"]
        n4["Submission-Spool"]
        n6["Logspec"]
        n7["KCIDB"]
        n8["Dashboard"]
  end
    n1["Maintainers"] -- send jobs --> n2
    n3["Labs"] <-- pull jobs --> n2
    n3 -- send results --> n2
    n2 -- send result submission --> n4
    n5 -- polls submissions --> n4
    n6 -- polls logs? --> n4
    n5 -- inserts --> n7
    n8 -- reads --> n7
    n6 -- sends incidents? --> n7
    n9["Users"] -- Access --> n8
    n5@{ shape: rect}
    n4@{ shape: docs}
    n6@{ shape: rect}
    n7@{ shape: db}
    n8@{ shape: rect}
    n1@{ shape: text}
    n3@{ shape: text}
    n9@{ shape: text}
     n6:::Rose
     n6:::Peach
    classDef Rose stroke-width:1px, stroke-dasharray:none, stroke:#FF5978, fill:#FFDFE5, color:#8E2236
    classDef Peach stroke-width:1px, stroke-dasharray:none, stroke:#FBB35A, fill:#FFEFDB, color:#8F632D

Loading

That first doc is only regarding the relations between "maintainers", "labs" and "KCIDB-Rest", right?

The hardware registry schema proposal is more impactful to the hardware listing definition. We can work on immediate improvements that don't require denormalization and also study other options while we see if that proposal will take effect. If so, we can change the hardware listing to use this new schema and do that with performance in mind from the ground up

[tales-aparecida]

That first doc is only regarding the relations between "maintainers", "labs" and "KCIDB-Rest", right?

Seems that way

The hardware registry schema proposal is more impactful to the hardware listing definition.
We can work on immediate improvements that don't require denormalization and also study other options while we see if that proposal will take effect.
If so, we can change the hardware listing to use this new schema and do that with performance in mind from the ground up

Let's bring this discussion to Monday's Dashboard WG meeting https://docs.google.com/document/d/1aoNpi69WRhn1nENbdAY1CPXYVkufbVnG_lFoGfA0aM4/edit?tab=t.0#bookmark=id.crlqtlaoyypz

[MarceloRobert]

Ref to #1584 and #1585