Skip to content

NOTES.md

Latest commit

 

History

History
81 lines (55 loc) · 4.02 KB

NOTES.md

File metadata and controls

81 lines (55 loc) · 4.02 KB

NOTES

Notes taken for research and development purposes.

Network Testing

Authorization

Iroh-docs comes with a builtin capability model with Write and Read operations per Namespace. In scoping the namespaces appropriately, this model is flexible enough for all our use-cases.

TODO: map out namespaces and scopes for each component

Consistency Considerations

A system with asynchronous processing implicates consistency trade-offs. The aim with this section is to surface and document these alongside their influence on current architectural decisions.

Audit trail

All admin operations, such as submitting an updated or initial configuration for a device, or changing the administrator constellation, are subject to be logged in a tamper-proof audit trail.

The audit trail requires to record all administrator operations in an append-only fashion across all coordinators. This forms an effective transaction boundary between the admin operation and the accompanying audit trail record.

High-Availability Coordinator (later)

Avoiding a single-point-of-failure in the coordinator functionality of the fleet is desired in the future.

The design can be guided by the PACELC design principle, which clearly delineated trade-offs under normal and partitioned networking conditions. We can distinguish different requirements for different types of data in the system.

The following list discusses various approaches and their trade-offs:

  • With the number of coordinators ≥ 3, it's feasible to use a distributed storage engine that relies on a consensus algorithm such as Raft to have immediate consistency. In this scenario there needs to be at least 2 coordinators online for write operations.
  • Using an eventual consistency model would allow even offline-write operations, at the cost of having intermittently inconsistent state. It's to be evaluated whether such a model can be a fit for the architecture requirements.

Persistence

Each fleet relies on persisted data for its operation which is at the Coordinator's responsibility.

  • Agents
    • Id
    • Type
    • Status
    • History of facts
    • Artifact graph
    • Reference to current position in the artifact graph
    • Metadata
      • Name?
      • Owner?
  • Enrolling Agents
    • Type
    • History of submitted facts
  • Enrolled Agents
    • Type
    • History of submitted facts
  • Credentials map
  • Audit logs

Artifacts

At first all artifacts are actually nix build outputs. It's worth considering that all artifacts could be content addressable and the fact that it's a nix build output could be irrelevant.

The artifact storage should allow to attach metadata to the artifact.

Question: is there any use-case that's not covered by a content-addressed store?

Could the content-addressed store be embedded in the coordinator process or should it run separately?

Nix Build Outputs

We certainly want to store Nix build outputs somewhere and need to make them retrievable by the Agent. As it's a requirement that the Agent doesn't have to evaluate its configuration, it's feasible to drop nix as a runtime dependency on the Agent altogether. This frees up the choice of protocols for transferring the artifacts, e.g. there's no need to stay within the limitations of the Nix Binary Cache protocol

Evaluation Snix' CAStore