Skip to content

Commit f2b0b65

Browse files
ecvecv
committed
feat: speak inventory v1alpha2 property-graph model
milo-os/inventory v0.1.0 ships a v1alpha2 API that collapses the typed per-kind CRDs into a property graph: generic Node + Edge (each with an attribute bag) under group graph.inventory.miloapis.com, with NodeType / EdgeType as the schema registry. Rework the plugin to speak it, replacing the v1alpha1 typed commands. Implements #9. Key changes: - Bump go.miloapis.com/inventory to v0.1.0; client + apply use the v1alpha2 scheme (graph.inventory.miloapis.com) - get <TYPE>: list nodes of an asset class with columns derived from the matching NodeType's attribute schema (falls back to the union of attribute keys present when no NodeType is registered) - get edges [--type/--from/--to]: list relationships - types: browse the NodeType/EdgeType registry - neighbors NODE [--edge/--direction]: graph traversal over edges - tree [--edge/--root-type]: containment hierarchy rebuilt from edges (default located-in, rooted at Region), with cycle guard - summary: counts per node type and per edge type - apply: graph kinds (NodeType, EdgeType, Node, Edge) in dependency order (types, then nodes, then edges); unsupported kinds rejected up front - Remove the v1alpha1 typed list commands (providers/regions/sites/ clusters/nodes)
1 parent 1fb5c6e commit f2b0b65

14 files changed

Lines changed: 693 additions & 555 deletions

‎README.md‎

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
# datumctl-inventory
22

3-
A [datumctl](https://github.com/datum-cloud/datumctl) plugin that provides a
4-
read view over the Datum Cloud physical inventory — providers, regions, sites,
5-
clusters, and nodes — served by the [milo inventory
6-
service](https://github.com/milo-os/inventory) (`inventory.miloapis.com/v1alpha1`).
7-
Once installed it is invoked as `datumctl inventory ...`.
3+
A [datumctl](https://github.com/datum-cloud/datumctl) plugin for the Datum Cloud
4+
inventory, modeled as a **property graph**: typed `Node`s (Region, Site,
5+
Cluster, Provider, Host, …) connected by typed `Edge`s (located-in, member-of,
6+
provided-by, …), each carrying an attribute bag. The available types and their
7+
attributes live in the `NodeType`/`EdgeType` schema registry. Served by the
8+
[milo inventory service](https://github.com/milo-os/inventory)
9+
(`graph.inventory.miloapis.com/v1alpha2`). Invoked as `datumctl inventory ...`.
810

911
## Install
1012

@@ -19,41 +21,40 @@ exposes it as `datumctl inventory`.
1921

2022
| Command | Description |
2123
|---|---|
22-
| `datumctl inventory providers` | List providers |
23-
| `datumctl inventory regions` | List regions |
24-
| `datumctl inventory sites [--region R] [--provider P]` | List sites |
25-
| `datumctl inventory clusters [--region R] [--site S]` | List clusters |
26-
| `datumctl inventory nodes [--region R] [--site S] [--cluster C]` | List nodes |
27-
| `datumctl inventory tree [--region R]` | region → site → node hierarchy |
28-
| `datumctl inventory summary` | Fleet-wide counts |
29-
| `datumctl inventory apply -f FILE [--dry-run=server]` | Create/update objects from a manifest |
24+
| `datumctl inventory get <TYPE>` | List nodes of an asset class; columns derived from the NodeType schema |
25+
| `datumctl inventory get edges [--type T] [--from N] [--to N]` | List edges (relationships) |
26+
| `datumctl inventory types` | List the NodeType/EdgeType schema registry |
27+
| `datumctl inventory neighbors NODE [--edge T] [--direction out\|in\|both]` | Nodes adjacent to NODE |
28+
| `datumctl inventory tree [--edge T] [--root-type T]` | Containment hierarchy from edges (default: `located-in`, rooted at `Region`) |
29+
| `datumctl inventory summary` | Counts per node type and edge type |
30+
| `datumctl inventory apply -f FILE [--dry-run=server]` | Create/update graph objects from a manifest |
3031

31-
The list subcommands accept `-o table|json|yaml` (default `table`).
32+
The `get`, `types`, `summary` commands accept `-o table|json|yaml` (default `table`).
3233

33-
`--region`, `--site`, and `--cluster` filter server-side using the
34-
`topology.inventory.miloapis.com/*` labels the inventory controllers propagate
35-
onto objects. `--provider` filters on the site's `providerRef`.
34+
Relationships that were typed fields in the old model (a site's region, a
35+
node's cluster) are now **edges** — query them with `get edges`, `neighbors`,
36+
or `tree` rather than as columns.
3637

3738
## Populating the inventory
3839

39-
`apply` is an idempotent, declarative upsert for inventory objects — for
40-
loading the inventory from declared configuration, not fleet management:
40+
`apply` is an idempotent, declarative upsert of graph objects — for loading the
41+
inventory from declared configuration, not fleet management:
4142

4243
```sh
43-
# Apply a manifest (objects land in dependency order: provider, region,
44-
# site, cluster, node — regardless of order in the file)
45-
datumctl inventory apply -f fleet.yaml
44+
# Apply a manifest (objects land in dependency order: node/edge types first,
45+
# then nodes, then edges — regardless of order in the file)
46+
datumctl inventory apply -f graph.yaml
4647

4748
# Pipe from a renderer
4849
render-fleet | datumctl inventory apply -f -
4950

5051
# Validate against the server without persisting
51-
datumctl inventory apply -f fleet.yaml --dry-run=server
52+
datumctl inventory apply -f graph.yaml --dry-run=server
5253
```
5354

5455
It uses server-side apply with field manager `datumctl-inventory`, so
55-
re-applying the same manifest makes no changes. Only `Provider`, `Region`,
56-
`Site`, `Cluster`, and `Node` are accepted.
56+
re-applying the same manifest makes no changes. Only `NodeType`, `EdgeType`,
57+
`Node`, and `Edge` are accepted.
5758

5859
Inventory objects are cluster-scoped on the Datum Cloud platform root, so the
5960
plugin talks to the platform API directly and takes no organization or project
@@ -65,7 +66,7 @@ datumctl injects context via environment variables and execs the plugin. The
6566
plugin reads `DATUM_API_HOST`, fetches a short-lived token through the
6667
credentials helper (`plugin.Token()`), and builds a controller-runtime client
6768
against the platform root using the milo inventory project's published typed
68-
API (`go.miloapis.com/inventory/api/v1alpha1`). See the
69+
API (`go.miloapis.com/inventory/api/v1alpha2`). See the
6970
[datumctl plugin docs](https://github.com/datum-cloud/datumctl/blob/main/docs/developer/plugins.md).
7071

7172
This split keeps Datum's CLI surface in `datum-cloud/` while depending on the

‎apply.go‎

Lines changed: 25 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,28 @@ package main
44

55
import (
66
"context"
7+
"encoding/json"
78
"fmt"
89
"io"
910
"os"
1011
"sort"
1112
"strings"
1213

1314
"github.com/spf13/cobra"
15+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
1416
"k8s.io/apimachinery/pkg/runtime"
1517
"k8s.io/apimachinery/pkg/runtime/serializer"
1618
utilyaml "k8s.io/apimachinery/pkg/util/yaml"
1719
"sigs.k8s.io/controller-runtime/pkg/client"
1820

19-
inventoryv1alpha1 "go.miloapis.com/inventory/api/v1alpha1"
21+
inventoryv1alpha2 "go.miloapis.com/inventory/api/v1alpha2"
2022
)
2123

2224
const fieldManager = "datumctl-inventory"
2325

24-
// applyOrder lists the inventory kinds apply handles, in dependency order:
25-
// parents are applied before the children that reference them.
26-
var applyOrder = []string{"Provider", "Region", "Site", "Cluster", "Node"}
26+
// applyOrder lists the graph kinds apply handles, in dependency order: the
27+
// type registry first, then nodes, then the edges that reference nodes.
28+
var applyOrder = []string{"NodeType", "EdgeType", "Node", "Edge"}
2729

2830
func kindOrder(kind string) (int, bool) {
2931
for i, k := range applyOrder {
@@ -39,14 +41,14 @@ func newApplyCmd() *cobra.Command {
3941
var dryRun string
4042
cmd := &cobra.Command{
4143
Use: "apply -f FILE",
42-
Short: "Create or update inventory objects from a manifest",
43-
Long: `Create or update inventory objects (providers, regions, sites, clusters,
44-
nodes) from a YAML or JSON manifest.
44+
Short: "Create or update inventory graph objects from a manifest",
45+
Long: `Create or update inventory graph objects (NodeType, EdgeType, Node, Edge)
46+
from a YAML or JSON manifest.
4547
4648
apply is an idempotent, declarative upsert: re-applying the same manifest makes
47-
no changes. Objects are applied in dependency order (providers, then regions,
48-
then sites, then clusters, then nodes) so a single mixed manifest lands cleanly.
49-
It uses server-side apply with field manager "datumctl-inventory".
49+
no changes. Objects are applied in dependency order (node/edge types first, then
50+
nodes, then the edges that reference them) so a single mixed manifest lands
51+
cleanly. It uses server-side apply with field manager "datumctl-inventory".
5052
5153
This is for populating the inventory from declared configuration — not fleet
5254
management. Inventory lives on the Datum Cloud platform root, so apply takes no
@@ -106,7 +108,7 @@ type applyObj struct {
106108
// kinds apply does not handle.
107109
func readManifests(stdin io.Reader, files []string) ([]applyObj, error) {
108110
scheme := runtime.NewScheme()
109-
if err := inventoryv1alpha1.AddToScheme(scheme); err != nil {
111+
if err := inventoryv1alpha2.AddToScheme(scheme); err != nil {
110112
return nil, fmt.Errorf("build scheme: %w", err)
111113
}
112114
decoder := serializer.NewCodecFactory(scheme).UniversalDeserializer()
@@ -130,15 +132,22 @@ func readManifests(stdin io.Reader, files []string) ([]applyObj, error) {
130132
if len(raw.Raw) == 0 {
131133
continue
132134
}
133-
obj, gvk, derr := decoder.Decode(raw.Raw, nil, nil)
134-
if derr != nil {
135+
// Check the kind before typed decode so an unsupported kind gets a
136+
// helpful message rather than the scheme's "not registered" error.
137+
var tm metav1.TypeMeta
138+
if derr := json.Unmarshal(raw.Raw, &tm); derr != nil {
135139
closeFn()
136-
return nil, fmt.Errorf("decode %s: %w", f, derr)
140+
return nil, fmt.Errorf("parse %s: %w", f, derr)
137141
}
138-
order, ok := kindOrder(gvk.Kind)
142+
order, ok := kindOrder(tm.Kind)
139143
if !ok {
140144
closeFn()
141-
return nil, fmt.Errorf("unsupported kind %q in %s (apply handles: %s)", gvk.Kind, f, strings.Join(applyOrder, ", "))
145+
return nil, fmt.Errorf("unsupported kind %q in %s (apply handles: %s)", tm.Kind, f, strings.Join(applyOrder, ", "))
146+
}
147+
obj, gvk, derr := decoder.Decode(raw.Raw, nil, nil)
148+
if derr != nil {
149+
closeFn()
150+
return nil, fmt.Errorf("decode %s: %w", f, derr)
142151
}
143152
co, ok := obj.(client.Object)
144153
if !ok {

‎apply_test.go‎

Lines changed: 36 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -9,35 +9,32 @@ import (
99
)
1010

1111
const sampleManifest = `
12-
apiVersion: inventory.miloapis.com/v1alpha1
13-
kind: Node
12+
apiVersion: graph.inventory.miloapis.com/v1alpha2
13+
kind: Edge
1414
metadata:
15-
name: node-a
15+
name: site-dfw1-in-uc
1616
spec:
17-
siteRef:
18-
name: us-central-1a
19-
hardware:
20-
cpuCores: 8
21-
cpuArchitecture: amd64
22-
memoryBytes: 1073741824
17+
type: located-in
18+
from:
19+
name: site-dfw1
20+
to:
21+
name: region-uc
2322
---
24-
apiVersion: inventory.miloapis.com/v1alpha1
25-
kind: Provider
23+
apiVersion: graph.inventory.miloapis.com/v1alpha2
24+
kind: NodeType
2625
metadata:
27-
name: netactuate
26+
name: Site
2827
spec:
29-
displayName: NetActuate
30-
type: Hosting
28+
displayName: Site
3129
---
32-
apiVersion: inventory.miloapis.com/v1alpha1
33-
kind: Site
30+
apiVersion: graph.inventory.miloapis.com/v1alpha2
31+
kind: Node
3432
metadata:
35-
name: us-central-1a
33+
name: site-dfw1
3634
spec:
37-
displayName: Dallas
38-
type: AvailabilityZone
39-
regionRef:
40-
name: us-central-1
35+
type: Site
36+
attributes:
37+
displayName: Dallas
4138
`
4239

4340
func TestReadManifestsParsesAndOrders(t *testing.T) {
@@ -49,14 +46,13 @@ func TestReadManifestsParsesAndOrders(t *testing.T) {
4946
t.Fatalf("got %d objects, want 3", len(objs))
5047
}
5148
sort.SliceStable(objs, func(i, j int) bool { return objs[i].order < objs[j].order })
52-
gotKinds := []string{objs[0].kind, objs[1].kind, objs[2].kind}
53-
want := []string{"Provider", "Site", "Node"}
49+
got := []string{objs[0].kind, objs[1].kind, objs[2].kind}
50+
want := []string{"NodeType", "Node", "Edge"}
5451
for i := range want {
55-
if gotKinds[i] != want[i] {
56-
t.Errorf("order[%d] = %s, want %s (full: %v)", i, gotKinds[i], want[i], gotKinds)
52+
if got[i] != want[i] {
53+
t.Errorf("order[%d] = %s, want %s (full: %v)", i, got[i], want[i], got)
5754
}
5855
}
59-
// GVK must be set on each object so server-side apply has apiVersion/kind.
6056
for _, o := range objs {
6157
if o.obj.GetObjectKind().GroupVersionKind().Kind == "" {
6258
t.Errorf("%s/%s missing GVK", o.kind, o.obj.GetName())
@@ -66,10 +62,10 @@ func TestReadManifestsParsesAndOrders(t *testing.T) {
6662

6763
func TestReadManifestsRejectsUnsupportedKind(t *testing.T) {
6864
const m = `
69-
apiVersion: inventory.miloapis.com/v1alpha1
70-
kind: Rack
65+
apiVersion: graph.inventory.miloapis.com/v1alpha2
66+
kind: Widget
7167
metadata:
72-
name: rack-a
68+
name: w
7369
`
7470
_, err := readManifests(strings.NewReader(m), []string{"-"})
7571
if err == nil || !strings.Contains(err.Error(), "unsupported kind") {
@@ -88,15 +84,18 @@ func TestReadManifestsEmpty(t *testing.T) {
8884
}
8985

9086
func TestKindOrder(t *testing.T) {
91-
if _, ok := kindOrder("Provider"); !ok {
92-
t.Error("Provider should be ordered")
87+
nt, ok := kindOrder("NodeType")
88+
if !ok {
89+
t.Fatal("NodeType should be ordered")
90+
}
91+
e, ok := kindOrder("Edge")
92+
if !ok {
93+
t.Fatal("Edge should be ordered")
9394
}
94-
p, _ := kindOrder("Provider")
95-
n, _ := kindOrder("Node")
96-
if !(p < n) {
97-
t.Errorf("Provider (%d) should sort before Node (%d)", p, n)
95+
if !(nt < e) {
96+
t.Errorf("NodeType (%d) should sort before Edge (%d)", nt, e)
9897
}
99-
if _, ok := kindOrder("Rack"); ok {
100-
t.Error("Rack should not be applyable")
98+
if _, ok := kindOrder("Widget"); ok {
99+
t.Error("Widget should not be applyable")
101100
}
102101
}

‎client.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ import (
1111
"sigs.k8s.io/controller-runtime/pkg/client"
1212

1313
"go.datum.net/datumctl/plugin"
14-
inventoryv1alpha1 "go.miloapis.com/inventory/api/v1alpha1"
14+
inventoryv1alpha2 "go.miloapis.com/inventory/api/v1alpha2"
1515
)
1616

1717
// newClient builds a controller-runtime client against the Datum Cloud platform
@@ -33,7 +33,7 @@ func newClient() (client.Client, error) {
3333
}
3434

3535
scheme := runtime.NewScheme()
36-
if err := inventoryv1alpha1.AddToScheme(scheme); err != nil {
36+
if err := inventoryv1alpha2.AddToScheme(scheme); err != nil {
3737
return nil, fmt.Errorf("build scheme: %w", err)
3838
}
3939

0 commit comments

Comments
 (0)