Skip to content

Commit 56dc8ed

Browse files
fregataaclaude
andcommitted
refactor(BA-7353): define the kernel payload beside the events that carry it
Review follow-up. A payload that only ever travels on a kernel event does not need a package of its own — it belongs with the other kernel event types. - Drop `common/interchange/` and define the payload in `events/event_types/kernel/types.py`, next to `KernelLifecycleEventReason`. Revert `common/AGENTS.md` and `common/KNOWLEDGE.md`, which only described that package, and leave `agent/resources.py` untouched. - Give each level of the occupancy its own model with a mapping field rather than nesting three dicts inline: `KernelOccupancy.devices` (by `DeviceName`) → `DeviceOccupancy.slots` (by `ResourceSlotName`) → `SlotOccupancy.amounts` (by `DeviceId`). A device supplies more than one slot when it is metered along more than one axis, as `cuda` does with `cuda.device` and `cuda.shares`. - Name it after what the manager does with it. It records the value as the kernel's occupancy, so the field is `occupancy` and the per-slot sum is `slot_totals`. - Key slots by `ResourceSlotName` throughout, not the legacy `SlotName`. - Declare the mappings as `Mapping`, so a consumer that tries to edit the payload it received is caught by the type checker. - Drop the field defaults: a producer states the whole payload. The exception is `DeviceCapacity`, whose two fields the compute plugin's own `ComputedDeviceCapacity` declares `NotRequired` — a device that measures neither reports neither. `slot_totals` stays a property rather than a `computed_field`: a computed field is written into the payload beside the occupancy it is derived from, where nothing reads it — a receiver recomputes it — and it can disagree with the value next to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent fefbdf6 commit 56dc8ed

11 files changed

Lines changed: 271 additions & 283 deletions

File tree

src/ai/backend/agent/resources.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ class KernelResourceSpec:
123123
slots: ResourceSlot
124124
"""Stores the original user-requested resource slots."""
125125

126-
allocations: MutableMapping[DeviceName, DeviceAllocation]
126+
allocations: MutableMapping[DeviceName, Mapping[SlotName, Mapping[DeviceId, Decimal]]]
127127
"""
128128
Represents the resource allocations for each slot (device) type and devices.
129129
"""

src/ai/backend/common/AGENTS.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,6 @@
2626
| `common/events/` | Event type definitions and dispatcher — see existing `AbstractEvent` subclasses |
2727
| `common/bgtask/` | Background task framework — extend `BaseBackgroundTaskHandler` |
2828
| `common/dto/` | Inter-component DTOs — see `common/dto/AGENTS.md` |
29-
| `common/interchange/` | Payloads one component sends another (event/RPC bodies) — see `common/interchange/AGENTS.md` |
30-
| `common/schema/` | Pydantic types the manager persists as a JSON DB column — see `common/schema/AGENTS.md` |
3129
| `common/exception.py` | Root `BackendAIError` and `ErrorCode` — all component exceptions inherit from here |
3230
| `common/types.py` | Common base types used across layers |
3331

src/ai/backend/common/KNOWLEDGE.md

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -27,20 +27,3 @@ types belong here is the core of this document.
2727
- The second criterion: types that, if placed in a higher layer, **would force an upward import** (`common/schema/` states this explicitly and is used only by the manager).
2828
- Single-consumer subpackages (`stage/` — agent-only, `resilience/` — manager-only) are not defects.
2929
- "It is convenient to put it somewhere neutral" is not a membership reason.
30-
31-
## Shared types are split by what crosses, not by who owns them
32-
33-
A type shared by two components still has to land in one of four places, and the
34-
question that separates them is what boundary it crosses:
35-
36-
| Package | Crosses |
37-
|---------|---------|
38-
| `interchange/` | A component boundary as a serialized body (event/RPC payload) |
39-
| `dto/` | An API boundary, per target component |
40-
| `schema/` | The persistence boundary — the manager stores it as a JSON column |
41-
| `data/` | Nothing; it is passed within a component |
42-
43-
`interchange/` exists because the first row had no home: `dto/` is organized by target
44-
component, and a payload has two sides, so neither side owns it. Its membership test is
45-
the consumer, not the producer — a field the receiving component never reads belongs in
46-
the producing component, not on the wire.

src/ai/backend/common/events/event_types/kernel/types.py

Lines changed: 114 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,19 @@
11
import enum
2-
from typing import Self
2+
from collections.abc import Mapping
3+
from decimal import Decimal
4+
from typing import Annotated, Self
5+
6+
from pydantic import Field
7+
8+
from ai.backend.common.identifier.resource_slot import ResourceSlotName
9+
from ai.backend.common.types import (
10+
BackendAISchema,
11+
ContainerId,
12+
DeviceId,
13+
DeviceName,
14+
ResourceSlotEntry,
15+
ServicePortProtocols,
16+
)
317

418

519
class KernelLifecycleEventReason(enum.StrEnum):
@@ -43,3 +57,102 @@ def from_value(cls, value: str | None) -> Self | None:
4357
except ValueError:
4458
pass
4559
return None
60+
61+
62+
type AllocationAmount = Annotated[
63+
Decimal,
64+
# An unbounded allocation is expressed as `Decimal("Infinity")`, which the default
65+
# `Decimal` constraint rejects. It survives the wire as the string "Infinity".
66+
Field(allow_inf_nan=True),
67+
]
68+
69+
70+
class SlotOccupancy(BackendAISchema):
71+
"""How much of one slot each individual device supplies."""
72+
73+
amounts: Mapping[DeviceId, AllocationAmount]
74+
75+
76+
class DeviceOccupancy(BackendAISchema):
77+
"""The slots one device supplies.
78+
79+
A device supplies more than one slot when it is metered along more than one axis —
80+
`cuda` supplies both `cuda.device` and `cuda.shares`.
81+
"""
82+
83+
slots: Mapping[ResourceSlotName, SlotOccupancy]
84+
85+
86+
class KernelOccupancy(BackendAISchema):
87+
"""
88+
The resources the kernel occupies, attributed to the devices supplying them.
89+
90+
`DeviceName` names a device (`cuda`) and `DeviceId` one of its units (`0`), so the
91+
two levels keyed by a device are not the same thing.
92+
"""
93+
94+
devices: Mapping[DeviceName, DeviceOccupancy]
95+
96+
@property
97+
def slot_totals(self) -> list[ResourceSlotEntry]:
98+
"""
99+
The per-device amounts summed per slot — what a caller records as occupancy.
100+
101+
A slot supplied by no device is left out rather than reported as zero, which is
102+
the distinction a caller storing the result depends on.
103+
104+
Not a `computed_field`: it would be written into the payload beside the
105+
occupancy it is derived from, and a receiver recomputes it anyway.
106+
"""
107+
totals: list[ResourceSlotEntry] = []
108+
for device in self.devices.values():
109+
for slot_name, slot in device.slots.items():
110+
if not slot.amounts:
111+
continue
112+
total = sum(slot.amounts.values(), Decimal(0))
113+
totals.append(ResourceSlotEntry(resource_type=slot_name, quantity=str(total)))
114+
return totals
115+
116+
117+
class DeviceCapacity(BackendAISchema):
118+
"""
119+
What a device reports about itself.
120+
121+
Both are defaulted, unlike every other field here: the compute plugin's own
122+
`ComputedDeviceCapacity` declares them `NotRequired`, so a device that measures
123+
neither reports neither.
124+
"""
125+
126+
mem: int | None = None
127+
proc: int | None = None
128+
129+
130+
class AttachedDevice(BackendAISchema):
131+
device_id: DeviceId
132+
model_name: str
133+
data: DeviceCapacity
134+
135+
136+
class ServicePortInfo(BackendAISchema):
137+
name: str
138+
protocol: ServicePortProtocols
139+
container_ports: list[int]
140+
host_ports: list[int | None]
141+
is_inference: bool
142+
143+
144+
class KernelCreationInfo(BackendAISchema):
145+
"""
146+
What the agent reports about a kernel once its container is up.
147+
148+
This is the contract with the manager, not a rendering of the agent's own resource
149+
spec: it carries the facts the manager records against the kernel and nothing else.
150+
"""
151+
152+
container_id: ContainerId
153+
kernel_host: str
154+
repl_in_port: int
155+
repl_out_port: int
156+
service_ports: list[ServicePortInfo]
157+
attached_devices: Mapping[DeviceName, list[AttachedDevice]]
158+
occupancy: KernelOccupancy

src/ai/backend/common/interchange/AGENTS.md

Lines changed: 0 additions & 34 deletions
This file was deleted.

src/ai/backend/common/interchange/CLAUDE.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

src/ai/backend/common/interchange/__init__.py

Whitespace-only changes.

src/ai/backend/common/interchange/kernel.py

Lines changed: 0 additions & 91 deletions
This file was deleted.

0 commit comments

Comments
 (0)