Skip to content

Commit f9dc4da

Browse files
fregataaclaude
andcommitted
refactor(BA-7353): define the kernel payload beside the events that carry it
Review follow-up. A payload that only ever travels on a kernel event does not need a package of its own — it belongs with the other kernel event types. - Drop `common/interchange/` and define the payload in `events/event_types/kernel/types.py`, next to `KernelLifecycleEventReason`. Revert `common/AGENTS.md` and `common/KNOWLEDGE.md`, which only described that package, and leave `agent/resources.py` untouched. - Give each level of the occupancy its own model with a mapping field rather than nesting three dicts inline: `KernelOccupancy.devices` (by `DeviceName`) → `DeviceOccupancy.slots` (by `ResourceSlotName`) → `SlotOccupancy.amounts` (by `DeviceId`). A device supplies more than one slot when it is metered along more than one axis, as `cuda` does with `cuda.device` and `cuda.shares`. - Name it after what the manager does with it. It records the value as the kernel's occupancy, so the field is `occupancy` and the per-slot sum is `slot_totals`. - Key slots by `ResourceSlotName` throughout, not the legacy `SlotName`. - Declare the mappings as `Mapping`, so a consumer that tries to edit the payload it received is caught by the type checker. - Drop the field defaults: a producer states the whole payload. The exception is `DeviceCapacity`, whose two fields the compute plugin's own `ComputedDeviceCapacity` declares `NotRequired` — a device that measures neither reports neither. - Let the default `Decimal` constraint reject a non-finite amount. `Infinity` belongs to a resource policy with `DefaultForUnspecified.UNLIMITED`, not to an allocation: a device supplies a finite share of what it has, and `alloc_map` never produces one. `slot_totals` stays a property rather than a `computed_field`: a computed field is written into the payload beside the occupancy it is derived from, where nothing reads it — a receiver recomputes it — and it can disagree with the value next to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent fefbdf6 commit f9dc4da

11 files changed

Lines changed: 270 additions & 282 deletions

File tree

src/ai/backend/agent/resources.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ class KernelResourceSpec:
123123
slots: ResourceSlot
124124
"""Stores the original user-requested resource slots."""
125125

126-
allocations: MutableMapping[DeviceName, DeviceAllocation]
126+
allocations: MutableMapping[DeviceName, Mapping[SlotName, Mapping[DeviceId, Decimal]]]
127127
"""
128128
Represents the resource allocations for each slot (device) type and devices.
129129
"""

src/ai/backend/common/AGENTS.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,6 @@
2626
| `common/events/` | Event type definitions and dispatcher — see existing `AbstractEvent` subclasses |
2727
| `common/bgtask/` | Background task framework — extend `BaseBackgroundTaskHandler` |
2828
| `common/dto/` | Inter-component DTOs — see `common/dto/AGENTS.md` |
29-
| `common/interchange/` | Payloads one component sends another (event/RPC bodies) — see `common/interchange/AGENTS.md` |
30-
| `common/schema/` | Pydantic types the manager persists as a JSON DB column — see `common/schema/AGENTS.md` |
3129
| `common/exception.py` | Root `BackendAIError` and `ErrorCode` — all component exceptions inherit from here |
3230
| `common/types.py` | Common base types used across layers |
3331

src/ai/backend/common/KNOWLEDGE.md

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -27,20 +27,3 @@ types belong here is the core of this document.
2727
- The second criterion: types that, if placed in a higher layer, **would force an upward import** (`common/schema/` states this explicitly and is used only by the manager).
2828
- Single-consumer subpackages (`stage/` — agent-only, `resilience/` — manager-only) are not defects.
2929
- "It is convenient to put it somewhere neutral" is not a membership reason.
30-
31-
## Shared types are split by what crosses, not by who owns them
32-
33-
A type shared by two components still has to land in one of four places, and the
34-
question that separates them is what boundary it crosses:
35-
36-
| Package | Crosses |
37-
|---------|---------|
38-
| `interchange/` | A component boundary as a serialized body (event/RPC payload) |
39-
| `dto/` | An API boundary, per target component |
40-
| `schema/` | The persistence boundary — the manager stores it as a JSON column |
41-
| `data/` | Nothing; it is passed within a component |
42-
43-
`interchange/` exists because the first row had no home: `dto/` is organized by target
44-
component, and a payload has two sides, so neither side owns it. Its membership test is
45-
the consumer, not the producer — a field the receiving component never reads belongs in
46-
the producing component, not on the wire.

src/ai/backend/common/events/event_types/kernel/types.py

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,18 @@
11
import enum
2+
from collections.abc import Mapping
3+
from decimal import Decimal
24
from typing import Self
35

6+
from ai.backend.common.identifier.resource_slot import ResourceSlotName
7+
from ai.backend.common.types import (
8+
BackendAISchema,
9+
ContainerId,
10+
DeviceId,
11+
DeviceName,
12+
ResourceSlotEntry,
13+
ServicePortProtocols,
14+
)
15+
416

517
class KernelLifecycleEventReason(enum.StrEnum):
618
AGENT_TERMINATION = "agent-termination"
@@ -43,3 +55,94 @@ def from_value(cls, value: str | None) -> Self | None:
4355
except ValueError:
4456
pass
4557
return None
58+
59+
60+
class SlotOccupancy(BackendAISchema):
61+
"""How much of one slot each individual device supplies."""
62+
63+
amounts: Mapping[DeviceId, Decimal]
64+
65+
66+
class DeviceOccupancy(BackendAISchema):
67+
"""The slots one device supplies.
68+
69+
A device supplies more than one slot when it is metered along more than one axis —
70+
`cuda` supplies both `cuda.device` and `cuda.shares`.
71+
"""
72+
73+
slots: Mapping[ResourceSlotName, SlotOccupancy]
74+
75+
76+
class KernelOccupancy(BackendAISchema):
77+
"""
78+
The resources the kernel occupies, attributed to the devices supplying them.
79+
80+
`DeviceName` names a device (`cuda`) and `DeviceId` one of its units (`0`), so the
81+
two levels keyed by a device are not the same thing.
82+
"""
83+
84+
devices: Mapping[DeviceName, DeviceOccupancy]
85+
86+
@property
87+
def slot_totals(self) -> list[ResourceSlotEntry]:
88+
"""
89+
The per-device amounts summed per slot — what a caller records as occupancy.
90+
91+
A slot supplied by no device is left out rather than reported as zero, which is
92+
the distinction a caller storing the result depends on.
93+
94+
Not a `computed_field`: it would be written into the payload beside the
95+
occupancy it is derived from, and a receiver recomputes it anyway.
96+
"""
97+
totals: list[ResourceSlotEntry] = []
98+
for device in self.devices.values():
99+
for slot_name, slot in device.slots.items():
100+
if not slot.amounts:
101+
continue
102+
total = sum(slot.amounts.values(), Decimal(0))
103+
totals.append(ResourceSlotEntry(resource_type=slot_name, quantity=str(total)))
104+
return totals
105+
106+
107+
class DeviceCapacity(BackendAISchema):
108+
"""
109+
What a device reports about itself.
110+
111+
Both are defaulted, unlike every other field here: the compute plugin's own
112+
`ComputedDeviceCapacity` declares them `NotRequired`, so a device that measures
113+
neither reports neither.
114+
"""
115+
116+
mem: int | None = None
117+
proc: int | None = None
118+
119+
120+
class AttachedDevice(BackendAISchema):
121+
device_id: DeviceId
122+
model_name: str
123+
data: DeviceCapacity
124+
125+
126+
class ServicePortInfo(BackendAISchema):
127+
name: str
128+
protocol: ServicePortProtocols
129+
container_ports: list[int]
130+
host_ports: list[int | None]
131+
is_inference: bool
132+
133+
134+
class KernelCreationInfo(BackendAISchema):
135+
"""
136+
What the agent reports about a kernel once its container is up.
137+
138+
This is the contract with the manager, not a rendering of the agent's own resource
139+
spec: it carries the facts the manager records against the kernel and nothing else.
140+
"""
141+
142+
container_id: ContainerId
143+
kernel_host: str
144+
repl_in_port: int
145+
repl_out_port: int
146+
service_ports: list[ServicePortInfo]
147+
attached_devices: Mapping[DeviceName, list[AttachedDevice]]
148+
occupancy: KernelOccupancy

src/ai/backend/common/interchange/AGENTS.md

Lines changed: 0 additions & 34 deletions
This file was deleted.

src/ai/backend/common/interchange/CLAUDE.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

src/ai/backend/common/interchange/__init__.py

Whitespace-only changes.

src/ai/backend/common/interchange/kernel.py

Lines changed: 0 additions & 91 deletions
This file was deleted.

0 commit comments

Comments
 (0)