refactor(BA-7315): carry event payloads as JSON instead of msgpack - #13746
Draft
fregataa wants to merge 2 commits into
Draft
refactor(BA-7315): carry event payloads as JSON instead of msgpack#13746fregataa wants to merge 2 commits into
fregataa wants to merge 2 commits into
Conversation
github-actions
Bot
added
size:XL
fregataa
added a commit
that referenced
this pull request
Aug 12, 2026
fregataa
marked this pull request as draft
August 12, 2026 15:26
Events are Pydantic models, so their fields already describe the body they carry. Publish that body as JSON via the final `to_message()` / `from_message()` pair and drop the hand-written `serialize()` / `deserialize()` tuple pair from every event, along with the msgpack wire format underneath it. - Remove 130 `serialize()` / `deserialize()` implementations and their abstract declarations; a positional tuple can no longer disagree with the field order. - Carry the body as a `payload` field on the wire and drop `args` / `legacy_body` along with the base64 wrapping broadcast needed for packed bytes. - Replace `creation_info: Mapping[str, Any]` with `KernelCreationInfo`, whose leaves are JSON-representable — the one payload that relied on pickled `ResourceSlot` / `Decimal` / `Path` values inside msgpack ext types. - Declare `creation_info` only on the started events, which are the only ones that ever carried it, and delete the manager's untyped duplicate view of it. - Raise `EventPayloadEncodingError` / `EventPayloadDecodingError` from the conversion pair, and ack a message whose body cannot be decoded instead of leaving it to be redelivered until the retry limit discards it. - Drop `args_matcher`, which matched against the positional tuple and had no callers. Every component reads and writes the same single stream and channel, so this requires the cluster to run one version across the upgrade. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`KernelResourceSpecData` described itself entirely in built-ins, so nothing in the payload said what a key or a mount actually was. - Type the fields as what they hold: `list[ResourceSlotEntry]` for slots, `dict[DeviceName, dict[ResourceSlotName, dict[DeviceId, str]]]` for allocations, and `list[MountData]` for mounts, which keeps a mount structured instead of flattening it to its `str()` form. `KernelCreationInfo` follows with `KernelId`, `ContainerId`, `DeviceId` and `DeviceName`. - `MountData` mirrors the agent's `Mount` and declares no defaults, so a producer states every field rather than inheriting one. - Omit a slot holding no device allocation from `to_resource_slot()` instead of recording it as zero. This restores what the manager helper it replaced did — a slot absent and a slot present at zero are not the same occupancy. - Drop `KernelResourceSpec.to_json_serializable_dict()` / `to_json()`, whose only caller was a log line, and `AgentRegistry.convert_resource_spec_to_resource_slot()`, which had no caller at all. Slot aggregation now has one implementation rather than three. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fregataa
force-pushed
the
refactor/BA-7315-json-event-payloads
branch
from
August 12, 2026 16:53
0bfd6b2 to
8a65349
Compare
fregataa
marked this pull request as ready for review
August 12, 2026 17:05
fregataa
marked this pull request as draft
August 13, 2026 01:43
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@finalto_message()/from_message()pair, and remove the 130 hand-writtenserialize()/deserialize()implementations plus the msgpack wire format beneath them. A positional tuple can no longer disagree with the field order it is built from.creation_info: Mapping[str, Any]with a typedKernelCreationInfowhose leaves are JSON-representable. This was the one payload relying on pickledResourceSlot/Decimal/Pathvalues carried in msgpack ext types, and it is now declared only on the started events — the only ones that ever carried it. The manager's untyped duplicate view of the same data is gone, and its allocation aggregation is nowKernelResourceSpecData.to_resource_slot().EventPayloadEncodingError/EventPayloadDecodingErrorfrom the conversion pair instead of letting Pydantic errors escape, and ack a message whose body cannot be decoded rather than leaving it to be redelivered until the retry limit discards it. Also dropsargs_matcher, which matched against the positional tuple and had no callers.Every component reads and writes the same single stream (
events) and channel (events_all), so this requires the cluster to run one version across the upgrade. In-flight msgpack entries are logged and skipped as malformed.This folds BA-7316 (removing the msgpack fallback) into this change: with the write path on JSON only, there is no fallback left to keep.
Test plan
pants test tests/unit/common/events:: tests/unit/common/message_queue::pants test tests/unit/common/test_events.py tests/unit/common/test_distributed.pypants test tests/unit/manager/repositories/scheduler::Resolves BA-7315