Merge remote-tracking branch 'origin/main' into bench-tooling-reorg

Author: dmorn

.scratch/moq-lite-04-protocol/PRD.md

@@ -0,0 +1,198 @@
+# MOQ Lite 04 protocol layer
+
+Status: done
+
+## Problem Statement
+
+`moqx` has a protocol-neutral QUIC transport foundation and now has initial MOQ
+Lite draft-04 message structs plus generic `MOQX.Codec` contracts. The next
+work is to turn those building blocks into a real MOQ Lite protocol layer.
+
+The transport PRD intentionally keeps "Full MOQ Lite implementation" out of
+scope. This PRD tracks the upper-layer work: MOQ Lite wire codecs, stream
+framing, and the protocol state machine that runs above `MOQX.Transport`.
+
+Relevant draft facts for this scope:
+
+- MOQ Lite draft-04 uses ALPN `moq-lite-04` for bare QUIC.
+- The session is active immediately after connection establishment; there is no
+  CLIENT_SETUP/SERVER_SETUP exchange.
+- Bidirectional transaction streams are used for Announce, Subscribe, Fetch,
+  Probe, and Goaway.
+- Publisher-created unidirectional Group Streams start with GROUP and then
+  carry FRAME messages.
+- Unknown stream types are reset for extension probing rather than treated as a
+  connection-level protocol failure.
+- Most messages carry a message length field, but that framing field is not
+  part of the semantic message struct.
+
+## Solution
+
+Build `MOQX.MOQLite04` as its own protocol variant on top of `MOQX.Transport`.
+
+The implementation should proceed in narrow slices:
+
+1. Define semantic message structs and stream type lookup helpers.
+2. Add shared binary helpers in `MOQX.Codec` for the integer/string/bytes
+   primitives needed by MOQ Lite and future draft-14 work.
+3. Implement MOQ Lite payload encoders and decoders for every message struct.
+4. Add a MOQ Lite stream codec that handles stream type prefixes, message
+   length fields, buffering, and payload decoder selection.
+5. Add a MOQ Lite session/stream state machine that enforces stream roles,
+   message ordering, graceful finish, abort sending, abort receiving, and
+   connection close behavior using `MOQX.Transport`.
+6. Add a URI-first `MOQX.MOQLite04.connect/2` facade that establishes a
+   transport connection and starts a role-neutral MOQ Lite client.
+7. Add a client runner that feeds normalized transport events into Session and
+   applies returned transport actions through `MOQX.Transport`.
+8. Add subscriber-style client operations, with role implied by each operation
+   rather than by a whole-connection mode.
+9. Add publisher-style client operations and a client-level
+   subscribe/group/frame smoke flow over `MOQX.Transport.Support`.
+
+## User Stories
+
+1. As a MOQ Lite client implementer, I want typed message structs so protocol
+   code can be written against Elixir values rather than ad hoc maps or raw
+   binaries.
+2. As a codec implementer, I want shared `MOQX.Codec` helpers so draft-14 and
+   MOQ Lite do not duplicate variable-length integer, string, and byte payload
+   parsing.
+3. As a protocol implementer, I want payload encoders and decoders to be
+   independent of stream buffering so they can be tested with simple binaries.
+4. As a session implementer, I want a stream codec that owns message lengths
+   and incomplete buffers so transport events can be converted into complete
+   messages deterministically.
+5. As a MOQ Lite endpoint, I want transaction stream state to reject invalid
+   message order, such as SUBSCRIBE_DROP before the first SUBSCRIBE_OK.
+6. As a subscriber, I want Fetch Streams to return FRAME messages directly on
+   the same bidirectional stream without a GROUP header.
+7. As a publisher, I want Group Streams to start with GROUP and then emit
+   ordered FRAME payloads until Finish Sending or Abort Sending.
+8. As a future draft-14 implementer, I want this work to avoid a common
+   protocol behaviour that would force draft-14 into MOQ Lite's stream model.
+
+## Implementation Decisions
+
+- `MOQX.Transport` remains the only transport boundary used by the protocol
+  layer.
+- Protocol code must not match raw `quicer` messages.
+- `MOQX.Codec` is generic. It must not contain MOQ Lite stream roles, message
+  atom tables, or session state.
+- `MOQX.MOQLite04` owns MOQ Lite draft-04 message structs, stream type lookup,
+  stream codec, and session state.
+- The upper protocol state machine is called a Session. Connection remains
+  transport vocabulary for `MOQX.Transport.Connection`.
+- Session APIs are pure reducers. `handle_transport/2` consumes normalized
+  `MOQX.Transport` events and `handle_command/2` consumes local application
+  intent.
+- Session reducers return protocol events and transport actions as data. They
+  do not call `MOQX.Transport`, `quicer`, or process APIs directly.
+- Transport stream identity is preserved. Stream-derived protocol events remain
+  tagged with a stream identity or ref, and each transport stream keeps its own
+  `MOQX.MOQLite04.StreamCodec` state.
+- `MOQX.MOQLite04.Error` is the protocol error boundary. Session code should
+  use structured errors and convert to integer transport application error
+  codes only when producing transport actions.
+- `StreamType` remains a typespec plus lookup helpers, not a struct.
+- Message structs contain semantic payload fields only. Message length is a
+  stream/message framing concern.
+- Decoder selection is stream-state dependent. A single decode-anything
+  function should not guess message type without context.
+- The first implementation targets native QUIC and ALPN `moq-lite-04`;
+  WebTransport remains out of scope.
+- Session APIs must receive explicit options or structs. Do not use
+  `Application` env as a test seam or mutable global configuration.
+- The future `MOQX.MOQLite04.connect/2` boundary should accept a URI
+  string or `URI.t()` directly. The module name already fixes the MOQ Lite
+  draft-04 variant, so a separate `MOQX.Endpoint` wrapper would duplicate
+  protocol/profile information at this layer.
+- Transport backend selection and backend options belong in explicit client
+  options, such as `transport: {MOQX.Transport.Quicer, opts}`, not in endpoint
+  data.
+
+## Testing Decisions
+
+Default tests should stay fast and deterministic.
+
+Initial coverage should include:
+
+- `MOQX.Codec` primitive round-trip and boundary tests;
+- payload encoder/decoder round-trip tests for every MOQ Lite message struct;
+- invalid enum, invalid length, incomplete buffer, and trailing-bytes errors;
+- stream codec tests for stream type prefixes and message length stripping;
+- pure state-machine tests for Announce, Subscribe, Fetch, Probe, Goaway, and
+  Group stream transitions;
+- support-transport tests showing the state machine consumes normalized
+  `MOQX.Transport` events rather than raw `quicer` events.
+
+Real QUIC or interop tests should stay tagged `:integration` and remain
+explicitly invoked.
+
+## Out of Scope
+
+- MOQT draft-14 protocol implementation.
+- WebTransport-over-HTTP/3 support.
+- Media payload parsing or codec/container awareness.
+- Production scheduling, caching, and relay fanout policy beyond the minimal
+  protocol state needed to exchange messages.
+- External MOQ Lite relay interoperability as a default unit-test requirement.
+
+## Progress
+
+Initial building block commit:
+
+- `3dc2a14 Add moq-lite message model and codec contracts`
+
+Delivered so far:
+
+- `MOQX.Codec`
+- `MOQX.Codec.Encoder`
+- `MOQX.Codec.Decoder`
+- `MOQX.Codec` varint, string, and byte payload helpers
+- `MOQX.MOQLite04` stream type lookup helpers
+- `MOQX.MOQLite04` subscribe response discriminator lookup helpers
+- `MOQX.MOQLite04` message structs for Announce, Subscribe, Fetch, Probe,
+  Goaway, Group, and Frame messages
+- `MOQX.MOQLite04` payload encoders and decoders for all message structs
+- `MOQX.MOQLite04.Error` structured protocol errors and transport application
+  code mapping
+- `MOQX.MOQLite04.StreamCodec` incremental receive and send framing for
+  opener and responder stream sides
+- `MOQX.MOQLite04.Session` pure reducer for normalized transport input,
+  local protocol commands, per-stream codec state, protocol events, and
+  transport action output
+- `MOQX.MOQLite04.connect/2` URI-first client facade over explicit transport
+  options, native QUIC ALPN `moq-lite-04`, and a role-neutral
+  `MOQX.MOQLite04.Client` value
+- `MOQX.MOQLite04.command/2` and `MOQX.MOQLite04.recv/2` client runner APIs
+  that thread the connected client value, keep `Session` pure, and apply
+  returned transport actions through `MOQX.Transport`
+- Role-neutral subscriber operations on `MOQX.MOQLite04` for
+  AnnounceInterest, Subscribe, SubscribeUpdate, Fetch, Probe, repeated Probe,
+  and Goaway transaction streams
+- Role-neutral publisher operations on `MOQX.MOQLite04` for accepting
+  peer-opened streams, sending Announce, SubscribeOk, and SubscribeDrop
+  responses, and publishing Group plus Frame messages on publisher-created
+  unidirectional streams
+- A client-level support-transport smoke flow that subscribes, accepts the
+  subscription, sends SubscribeOk, publishes a Group, and delivers the original
+  Frame payload through public client APIs
+- Session tests for Announce, Subscribe, Fetch, Probe, Goaway, Group,
+  stream lifecycle, support transport normalized events, and the
+  reference-style subscribe/group/frame smoke flow
+
+Follow-up integration work:
+
+- issue 09: real QUIC MOQ Lite 04 integration tests over
+  `MOQX.Transport.Quicer`
+
+## References
+
+- <https://datatracker.ietf.org/doc/html/draft-lcurley-moq-lite-04>
+- <https://datatracker.ietf.org/doc/draft-ietf-moq-transport/14/>
+- `docs/adr/0001-transport-boundary-support-transport-and-benchmark-harness.md`
+- `docs/adr/0002-native-quic-first-webtransport-out-of-scope.md`
+- `docs/adr/0003-validated-endpoints-above-raw-transport.md`
+- `docs/adr/0006-protocol-variants-own-session-state-codec-stays-generic.md`
+- `docs/adr/0007-protocol-sessions-are-pure-reducers.md`

.scratch/moq-lite-04-protocol/issues/01-add-message-model-and-codec-contracts.md

@@ -0,0 +1,44 @@
+> *This was generated by AI during triage.*
+
+# Add MOQ Lite 04 message model and generic codec contracts
+
+Status: closed
+Type: enhancement
+
+## Parent
+
+`.scratch/moq-lite-04-protocol/PRD.md`
+
+## What to build
+
+Add the initial protocol building blocks needed before implementing MOQ Lite
+wire encoding or session state.
+
+## Acceptance criteria
+
+- [x] `MOQX.Codec` exists as a generic namespace for shared codec helpers.
+- [x] `MOQX.Codec.Encoder` exists as an Elixir protocol for typed
+      value-to-iodata encoding.
+- [x] `MOQX.Codec.Decoder` exists as a generic behaviour for complete payload
+      decoding.
+- [x] `MOQX.MOQLite04` exists as the MOQ Lite draft-04 namespace.
+- [x] MOQ Lite draft-04 stream type lookup works in both directions.
+- [x] Stream type is modeled as a typespec plus lookup tables, not a struct.
+- [x] MOQ Lite draft-04 messages are represented as semantic structs.
+- [x] Message length is kept out of the message structs.
+- [x] Tests cover the codec contracts, stream type lookup, and message structs.
+
+## Progress
+
+Implemented in commit `3dc2a14 Add moq-lite message model and codec contracts`.
+
+Validation completed before commit:
+
+- `mix format`
+- `mix test`
+- `mix credo --strict`
+
+## Comments
+
+- 2026-06-05: Closed after landing the generic `MOQX.Codec` contracts and the
+  `MOQX.MOQLite04` message model.

.scratch/moq-lite-04-protocol/issues/02-add-shared-binary-codec-helpers.md

@@ -0,0 +1,59 @@
+> *This was generated by AI during triage.*
+
+# Add shared binary codec helpers
+
+Status: closed
+Type: enhancement
+
+## Parent
+
+`.scratch/moq-lite-04-protocol/PRD.md`
+
+## What to build
+
+Add protocol-neutral binary helpers under `MOQX.Codec` for the primitive wire
+formats needed by MOQ Lite draft-04 payload and stream codecs.
+
+The helpers should be small pure functions, not a process or protocol-specific
+dispatcher.
+
+## Acceptance criteria
+
+- [x] Variable-length integer encode/decode helpers exist and return explicit
+      `{:ok, value, rest}` or `{:error, reason}` results for decode.
+- [x] Boundary tests cover every encoded integer size class required by the
+      referenced MOQT-family drafts.
+- [x] Incomplete integer inputs return a distinguishable incomplete-data error.
+- [x] Length-prefixed UTF-8 string helpers exist for MOQ Lite `(s)` fields.
+- [x] Length-prefixed byte payload helpers exist for MOQ Lite `(b)` fields.
+- [x] Helpers reject malformed lengths instead of reading past available data.
+- [x] The module remains protocol-neutral: no MOQ Lite message atoms, stream
+      roles, or session state in `MOQX.Codec`.
+- [x] Tests are pure unit tests and do not require live QUIC transport.
+
+## Notes
+
+Consult MOQ Lite draft-04 encoding sections before implementing the exact
+integer, string, and byte helper semantics. Keep helper names generic enough to
+serve future MOQT draft-14 codec work.
+
+## Progress
+
+Implemented in the current working tree:
+
+- `MOQX.Codec.encode_varint/1`
+- `MOQX.Codec.decode_varint/1`
+- `MOQX.Codec.encode_string/1`
+- `MOQX.Codec.decode_string/1`
+- `MOQX.Codec.encode_bytes/1`
+- `MOQX.Codec.decode_bytes/1`
+
+Validation:
+
+- `mix test test/moqx/codec/binary_test.exs`
+- `mix test test/moqx/codec/binary_test.exs test/moqx/moq_lite_04_test.exs test/moqx/moq_lite_04/payload_codec_test.exs test/moqx/codec/encoder_decoder_test.exs`
+
+## Comments
+
+- 2026-06-05: Closed after adding generic QUIC varint and length-prefixed
+  string/bytes helpers under `MOQX.Codec`.

.scratch/moq-lite-04-protocol/issues/03-implement-moq-lite-payload-encoders-and-decoders.md

@@ -0,0 +1,63 @@
+> *This was generated by AI during triage.*
+
+# Implement MOQ Lite 04 payload encoders and decoders
+
+Status: closed
+Type: enhancement
+
+## Parent
+
+`.scratch/moq-lite-04-protocol/PRD.md`
+
+## What to build
+
+Implement MOQ Lite draft-04 payload encoding and decoding for the message
+structs under `MOQX.MOQLite04`.
+
+This slice should produce payload bytes for already-selected message shapes and
+parse complete payload bytes back into typed structs. Stream type prefixes,
+message length fields, buffering, and stream-state dispatch are handled by a
+later stream codec slice.
+
+## Acceptance criteria
+
+- [x] `MOQX.Codec.Encoder` implementations exist for all MOQ Lite draft-04
+      message structs.
+- [x] Concrete decoders exist for all MOQ Lite draft-04 message payload shapes.
+- [x] Round-trip tests cover AnnounceInterest, Announce, Subscribe,
+      SubscribeUpdate, SubscribeOk, SubscribeDrop, Fetch, Probe, Goaway, Group,
+      and Frame.
+- [x] Invalid status/order/discriminator values return explicit errors.
+- [x] Decoders reject trailing bytes when the caller expects a complete payload.
+- [x] Message length remains outside the message structs and outside payload
+      encoder implementations unless the stream codec explicitly adds it.
+- [x] Subscribe response type discriminators are handled without adding a
+      synthetic `type` field to `SubscribeOk` or `SubscribeDrop` structs.
+- [x] Tests do not depend on `Application` env or live QUIC sockets.
+
+## Notes
+
+Use `MOQX.Codec` helpers for primitive parsing. Keep decoder selection
+contextual: the stream codec or state machine should decide which decoder is
+valid next.
+
+## Progress
+
+Implemented in the current working tree:
+
+- `MOQX.Codec.Encoder` implementations for every `MOQX.MOQLite04` message
+  struct.
+- `MOQX.Codec.Decoder` implementations through each message module's
+  `decode/2` callback.
+- Subscribe response discriminator lookup helpers on `MOQX.MOQLite04`.
+
+Validation:
+
+- `mix test test/moqx/moq_lite_04/payload_codec_test.exs`
+- `mix test test/moqx/codec/binary_test.exs test/moqx/moq_lite_04_test.exs test/moqx/moq_lite_04/payload_codec_test.exs test/moqx/codec/encoder_decoder_test.exs`
+
+## Comments
+
+- 2026-06-05: Closed after adding payload-only encoders/decoders. Stream type
+  prefixes, message length fields, buffering, and stream-state dispatch remain
+  for issue 04.