Skip to content

architecture.md

Latest commit

 

History

History
599 lines (442 loc) · 37.7 KB

architecture.md

File metadata and controls

599 lines (442 loc) · 37.7 KB
Raw

Architecture

This document describes the internal structure of stamp-suite: the module layout, the receiver backends, the packet-processing pipeline, the TLV reference, and the optional observability subsystems (Prometheus, SNMP). The top-level README covers user-facing material — install, run, configure. Anything that explains how the implementation works lives here.

Overview

┌─────────────────┐         UDP/862         ┌─────────────────┐
│ Session-Sender  │ ──────────────────────► │Session-Reflector│
│   (stamp-suite) │ ◄────────────────────── │  (stamp-suite -i)│
└─────────────────┘    Reflected Packets    └─────────────────┘

A single binary plays both roles. The Session-Sender transmits STAMP test packets and timestamps the reply; the Session-Reflector receives, timestamps, and reflects. STAMP wire formats follow RFC 8762; optional TLV extensions follow RFC 8972, with additions from RFC 9503 (Segment Routing), RFC 9534 (Link Aggregation Group micro-sessions), and two active IETF drafts.

Module Structure

  • main.rs — Entry point and CLI handling. Branches into sender or reflector mode based on --is-reflector.
  • configuration.rs — Command-line argument parsing (clap derive), TOML config-file merging, validation. Auth-mode helpers (is_auth, is_enc, is_open).
  • packets.rs — STAMP packet structures for authenticated and unauthenticated modes. Big-endian fixed-width serialization via bincode.
  • sender.rs — Session-Sender implementation: packet assembly, send loop, RTT statistics.
  • receiver/ — Session-Reflector implementations:
    • receiver/mod.rs — Shared STAMP-level pipeline. All TLV parsing, HMAC verification, Return Path handling, session tracking, and counter updates live here. Both backends call into the same process_stamp_packet after capturing a packet.
    • receiver/nix.rs — Default backend on Linux and macOS. Uses a tokio::net::UdpSocket with IP_RECVTTL, IP_RECVTOS, and IP_PKTINFO (plus IPv6 equivalents) to extract per-packet metadata via recvmsg control messages.
    • receiver/pnet.rs — Default backend on Windows; opt-in elsewhere via --features ttl-pnet. Captures at the datalink layer via libpcap / Npcap.
  • session.rs — SessionManager and per-session state. Atomic sequence-number generation, idle-timeout reaping.
  • time.rs — Timestamp generation in NTP and PTP formats.
  • clock_format.rs — ClockFormat enum (NTP / PTP) with parsing.
  • stamp_modes.rs — STAMP mode enum (Authenticated / Unauthenticated).
  • tlv/ — TLV extension support (RFC 8972 + 9503 + 9534 + drafts). Subdivided by TLV type with shared parsing scaffolding.
  • crypto.rs — HMAC computation and verification (compute_packet_hmac, verify_packet_hmac, HmacKey).
  • metrics/ — Prometheus metrics (optional, requires metrics feature):
    • sender_metrics.rs — Sender-side metrics
    • reflector_metrics.rs — Reflector-side metrics
  • snmp/ — SNMP AgentX sub-agent (optional, requires snmp feature, Unix only):
    • agentx.rs — AgentX protocol implementation (RFC 2741)
    • handler.rs — STAMP-SUITE-MIB handler
    • oids.rs — OID constants
    • state.rs — Shared state types

Receiver Backends

The reflector has two receive-path backends. They differ in how they extract per-packet metadata (TTL, DSCP/ECN, destination address, raw IP headers), not in STAMP protocol behaviour — both go through the same process_stamp_packet pipeline.

nix backend (default on Linux and macOS)

Binds a normal tokio::net::UdpSocket and attaches IP_RECVTTL, IP_RECVTOS, and IP_PKTINFO (with IPv6 equivalents) so the kernel hands per-packet metadata through recvmsg control messages. The kernel performs UDP demultiplexing and checksum validation; userspace only sees traffic destined for the bound port.

pnet backend (default on Windows, opt-in elsewhere)

Captures frames at the datalink layer via libpcap / Npcap, parses Ethernet / IPv4 / IPv6 / UDP manually, and sends replies through a separate UdpSocket. Sees full IP headers, including IPv6 extension headers.

Why nix is the default where it works

Picking pnet everywhere would simplify the codebase slightly (one capture loop instead of two), but it would regress every Linux/macOS deployment on several independent axes:

  1. Privileges. The nix backend runs as an unprivileged user — it binds a UDP port and that is it. The pnet backend needs CAP_NET_RAW (or setcap cap_net_raw=eip on the binary, or plain root). That matters for container images, systemd hardening, CI runners, SaaS deployments, and anywhere security policy limits capabilities.

  2. Runtime dependencies. nix only needs libc. pnet links against libpcap on Unix and Npcap on Windows; these must be installed out-of-band before the binary will start. A statically-linked nix build drops into minimal containers and immutable OS images without extra packaging work.

  3. Kernel packet filtering. With nix, the kernel demultiplexes UDP by destination port before userspace wakes up — stamp-suite only ever sees its own traffic. With pnet, every frame on the interface reaches userspace; we then discard everything that is not UDP to our port. On a 10 Gb/s link carrying unrelated traffic this wastes CPU and causes scheduling jitter that pollutes delay measurements.

  4. Interaction with the host firewall. nix goes through the normal socket path, so iptables / nftables INPUT rules and per-socket accounting behave exactly as the operator expects. pnet bypasses INPUT on receive and can bypass OUTPUT on raw send, so host-level policy is silently skipped.

  5. Kernel does the heavy lifting. nix lets the kernel handle UDP checksum, fragmentation, path MTU discovery, ICMP unreachable, ARP / NDP for the next hop, and routing-table changes. A pnet-everywhere design would have to re-implement or work around each of these.

  6. Observability. The nix backend has a real socket visible to ss -u, netstat, lsof, tracing tools, and systemd socket accounting. The pnet backend has none of these handles.

Tradeoff

Keeping two backends has a real cost: packet-processing changes that touch the capture path have to be mirrored in both files. We mitigate this by keeping all STAMP-level logic (TLV parsing, HMAC, Return Path handling, session tracking, counter updates) in receiver/mod.rs — the two backends differ only in how they capture packets and whether they send over an async tokio socket or a blocking std socket.

The other consequence is that a handful of features that genuinely require raw IP-header visibility — currently just the Reflected Fixed Header Data (Type 247) and Reflected IPv6 Extension Header Data (Type 246) TLVs from draft-ietf-ippm-stamp-ext-hdr — are only populated on the pnet backend. On the nix backend the reflector echoes the TLV with the U-flag set per RFC 8972 §4.2, and logs a one-time warning suggesting a rebuild with --features ttl-pnet if header reflection is actually needed. This follows the draft's own "may be unsupported by the reflector" semantics, so the sender sees a spec-compliant response either way.

If you specifically need TLV 246/247 reflection, or you want to craft outgoing packets with non-default IPv6 extension headers, build with --no-default-features --features ttl-pnet and accept the tradeoffs above.

Packet Processing Pipeline

Both backends, after capturing a packet, hand it to the same shared pipeline in receiver/mod.rs:

  1. Parse — Decode the STAMP base header into PacketUnauthenticated or PacketAuthenticated.
  2. HMAC verify — In authenticated mode, crypto::verify_packet_hmac checks the keyed digest over the base packet. Failures increment hmac_failures_total and drop the packet.
  3. TLV pipeline — Walk the TLV chain. For each known type, run its typed parser; unknown types are preserved and echoed with the U-flag set (per RFC 8972 §4.2).
  4. Session lookup / update — Per-client counters (reflector_rx_count, reflector_tx_count, last_reflection) are always tracked. If --stateful-reflector is set, a per-client sequence number is also assigned via SessionManager.
  5. RFC 9503 processing — Destination Node Address matching against local_addresses; Return Path action selection (Normal, SuppressReply, AlternateAddress, Srv6Forward, UnsupportedSr). Encoded into a ReturnPathAction carried in StampResponse; the send path attempts best-effort SRv6 SRH forwarding for Srv6Forward (see Return Path TLV below).
  6. Assemble reply — assemble_unauth_answer_with_tlvs / assemble_auth_answer_with_tlvs build the response, populate reflector-side TLV fields (DM counters, Follow-Up Telemetry, Timestamp Info, Location, Class of Service, etc.), and recompute HMACs (base + TLV) if applicable.
  7. Send — Reply to the original source, an alternate address (Return Path), or suppress entirely.

The ProcessingContext struct carries per-packet shared state (counters, optional SessionManager reference, local addresses, sender port). ReceiverSharedState (counters, session manager, start time) lives at the receiver level and is created once via create_shared_state() before run_receiver().

Operational Characteristics

A few cross-cutting operational invariants are worth pinning down separately, since they affect every code path that touches the network or the optional subsystems.

Packet-receive contract: --strict-packets

The reflector's packet-parse path has two modes:

  • Lenient (default) — short packets are zero-filled to the canonical size per RFC 8762 §4.6, then parsed. HMAC, when present, is verified against the canonical (zero-padded) buffer. This is the interop-friendly mode and matches the behaviour TWAMP-Light senders expect.
  • Strict (--strict-packets) — short packets are rejected at the parser. The HMAC, MBZ, and require_hmac checks are independent of strictness — strict mode only changes how short packets are treated.

The contract is exhaustively pinned by strict_packets_* tests in src/receiver/mod.rs, including the explicit RFC 8762 §4.1.1 case that non-zero MBZ on receipt is always ignored in both modes (the RFC mandates "MUST be ignored on receipt"). Both modes also tolerate a zero-byte buffer without panicking.

Capture-thread liveness signal

ReceiverSharedState carries capture_alive: Arc<AtomicBool> (initialised true). Both backends clear this flag when their receive loop exits unexpectedly (nix: socket creation or bind failure; pnet: missing interface, channel-init failure, send-socket bind failure, or a spawn_blocking panic propagated up through the JoinHandle). The flag exists so a future /healthz endpoint (and external monitors today, via SNMP or signal) can distinguish "process alive but not reflecting" from "process alive and healthy" without scraping stdout. Operationally this means a single dead capture loop never goes silent — it surfaces as false on this flag and as log::error! lines in the journal.

Observability subsystem failure semantics (--metrics vs --snmp)

The two optional subsystems handle initialisation failure asymmetrically by design:

  • --metrics fails fast. If the operator explicitly requested a Prometheus endpoint and the bind fails (AddrInUse, AddrNotAvailable, PermissionDenied, …), main.rs exits with a specific error message naming the io::ErrorKind. The reasoning: silently disabling the endpoint would leave dashboards and alerts running blind without any signal that they are.
  • --snmp degrades gracefully. If the AgentX master socket is absent or unreachable (e.g. net-snmpd hasn't started yet during boot), main.rs logs a warning and continues with None. The reflector's primary duty — forwarding STAMP packets — is unaffected by the SNMP sub-agent being down. Operators who want SNMP-required-to-start semantics can wrap stamp-suite.service with a systemd ordering directive (After=snmpd.service, Requires=snmpd.service).

The same asymmetry is documented for end-users in usage.md.

AgentX sub-agent panic-resistance

The AgentX event loop runs inside tokio::task::spawn_blocking. A separate supervisor tokio::spawn task awaits the JoinHandle and logs panics (JoinError::is_panic()) and abnormal terminations rather than dropping them silently. The decoder itself was audited for production-path panics; every buffer-indexing site (agentx::decode_header, decode_oid, decode_search_range, AgentXSession::handle_get_bulk) is preceded by an explicit length check returning AgentXError::Protocol. The MibHandler dispatch (StampMibHandler::get/get_next) bounds-checks OIDs via Oid::starts_with before any indexing. Coverage is locked in by the malformed-input tests in src/snmp/agentx.rs and src/snmp/handler.rs.

Session Management

SessionManager is always instantiated, regardless of the --stateful-reflector flag. The flag only controls one thing: whether the assembler uses per-client sequence numbering (ProcessingContext.session_manager: Option<&Arc<SessionManager>>) instead of a global counter. Per-client packet counters and last-reflection tracking — needed by the Direct Measurement (Type 5) and Follow-Up Telemetry (Type 7) TLVs — run unconditionally because the TLV semantics require them.

Sessions are reaped after --session-timeout seconds of inactivity (default 300 s). When the SNMP feature is enabled, SessionSummary / session_summaries_extended() exposes per-session data for the SNMP session table.

TLV Extensions Reference

The implementation supports RFC 8972 TLV (Type-Length-Value) extensions, which allow STAMP packets to carry optional data beyond the base packet format.

Supported TLV Types

Status labels used in this table — kept aligned with the (forthcoming) standards matrix:

  • supported — structured parsing, validation, and reflector-side field population are complete and conform to the spec.
  • partial — implemented to the spec on most paths but with a named gap (sub-TLV, sub-field, or backend-restricted feature). The gap is explicit in the table row.
  • experimental — implements an active IETF draft. Wire format or type number may change before standardisation; treat as best-effort interop only.
  • interop-only — present solely to interoperate with another implementation's non-standard extension. Off in default builds.
Type Name Description Status
1 Extra Padding Can carry Session-Sender ID (SSID) in first 2 bytes supported
2 Location Source/destination addresses and ports (RFC 8972 §4.2) supported
3 Timestamp Info Sync source and timestamping method (RFC 8972 §4.3) supported
4 Class of Service DSCP/ECN measurement (RFC 8972 §5.2) supported
5 Direct Measurement Sender/reflector packet counters (RFC 8972 §4.5) supported
6 Access Report Access identifier and return code (RFC 8972 §4.6) supported
7 Follow-Up Telemetry Previous reflection seq/timestamp (RFC 8972 §4.7) supported
8 HMAC TLV integrity verification (must be last) supported
9 Destination Node Address Verify intended reflector identity (RFC 9503 §4) supported
10 Return Path Control reply routing: suppress, alternate address, SR-MPLS, SRv6 (RFC 9503 §5) supported — suppress / alternate address / SRv6 best-effort SRH forwarding (opt-in --srv6-return-forwarding, Linux+IPv6, graceful U-flag fallback); SR-MPLS echoed with U-flag (out of scope for userspace UDP)
11 Micro-session ID LAG member link identifiers for per-link measurement (RFC 9534 §3.1) supported
12 Reflected Test Packet Control Asymmetrical reply request — count, length, interval (draft-ietf-ippm-asymmetrical-pkts-14, IANA-assigned) supported — emission, length padding (up to --reflected-control-max-size), L3 Address Group sub-TLV match; L2 sub-TLV present sets U-flag on backends without MAC visibility
240 BER Bit Pattern in Padding Repeated bit pattern carried alongside Extra Padding (draft-gandhi-ippm-stamp-ber-05) experimental
241 BER Bit Error Count u32 error-bit count, computed by reflector experimental
242 BER Max Bit Error Burst Size u32 longest consecutive error run, computed by reflector experimental — wire-format collision with teaparty Heartbeat (same Type 242); see note below
246 Reflected IPv6 Extension Header Data Reflects received IPv6 Hop-by-Hop / Destination Options headers (draft-ietf-ippm-stamp-ext-hdr) partial — pnet backend only (nix backend echoes with U-flag)
247 Reflected Fixed Header Data Reflects the raw 20-byte IPv4 or 40-byte IPv6 fixed header (draft-ietf-ippm-stamp-ext-hdr) partial — pnet backend only (nix backend echoes with U-flag)

IANA registry: Type 12 and the C flag (bit 3 of TLV flags) are IANA-assigned per draft-ietf-ippm-asymmetrical-pkts-14. Types 240–251 are Experimental Use per RFC 8972 §6 — picks by individual implementations.

Type 242 collision: stamp-suite uses Type 242 for BER Max Bit Error Burst Size (draft-gandhi-ippm-stamp-ber-05); teaparty uses the same Type 242 for an experimental Heartbeat TLV. Both are within the Experimental Use range so neither is wrong per IANA, but the wire formats are mutually incompatible. Until an explicit experimental-teaparty-compat build path exists, deployments that mix the two implementations should disable BER on stamp-suite or Heartbeat on teaparty rather than relying on which one wins the byte race.

Backend restriction on Types 246/247: Both require the reflector to copy raw IP-header bytes into the response, which is only possible when the capture path sees full IP headers. The default nix UDP-socket backend cannot provide this — see Receiver Backends for why the default remains nix. On the nix backend these TLVs are echoed with the U-flag set per RFC 8972 §4.2 and a one-time warning is logged.

TLV Handling Modes

The reflector supports two TLV handling modes via --tlv-mode:

Mode Behavior
echo (default) Echo TLVs back to sender, marking unknown types with U-flag
ignore Strip all TLVs from response (backward compatibility)

Backward Compatibility

The implementation is fully backward compatible:

  • No TLVs in packet: Standard RFC 8762 handling is used
  • TLVs present: Handled according to --tlv-mode setting
  • Old clients: Work seamlessly with TLV-enabled reflectors
  • New clients with SSID: Reflectors without TLV support will zero-pad the response

TLV Wire Format (RFC 8972 Section 4.2)

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|STAMP TLV Flags|     Type      |            Length             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         Value...                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  • Flags (1 octet): U=Unrecognized (bit 0), M=Malformed (bit 1), I=Integrity failed (bit 2), C=Conformant Reflected Packet (bit 3, draft-ietf-ippm-asymmetrical-pkts, set only on Type 12 TLVs), Reserved (bits 4-7)
  • Type (1 octet): TLV type identifier (0-255)
  • Length (2 octets): Length of Value field in bytes

Class of Service TLV (RFC 8972 §5.2)

The CoS TLV enables measurement of DSCP and ECN handling across the network path:

# Sender requests DSCP 46 (EF) and ECN 2
stamp-suite --remote-addr 192.168.1.100 --cos --dscp 46 --ecn 2

The reflector automatically fills in:

  • DSCP2/ECN2: Values received at the reflector's ingress
  • RP (Reverse Path): Set to 1 if local policy rejected the requested DSCP

This allows detection of DSCP remarking or ECN modification in the network.

Location TLV (RFC 8972 §4.2)

The Location TLV reports the observed source and destination addresses and ports at the reflector:

stamp-suite --remote-addr 192.168.1.100 --location

The reflector fills in the actual destination IP from the received packet (using IP_PKTINFO/IPV6_RECVPKTINFO on nix, or parsed IP headers on pnet), so it reports the correct address even when bound to a wildcard (0.0.0.0/::). Address information is carried as sub-TLVs (IPv4 or IPv6 source/destination).

Direct Measurement TLV (RFC 8972 §4.5)

The Direct Measurement TLV carries per-session packet counters for loss measurement:

stamp-suite --remote-addr 192.168.1.100 --direct-measurement
  • Sender fills its transmit count (incremented per packet)
  • Reflector fills its receive and transmit counts for the client's session

Counters are tracked per-client regardless of whether --stateful-reflector is enabled.

Follow-Up Telemetry TLV (RFC 8972 §4.7)

The Follow-Up Telemetry TLV carries information about the previously reflected packet:

stamp-suite --remote-addr 192.168.1.100 --follow-up-telemetry

The reflector fills in the sequence number and timestamp from the last reflection for the client's session, along with its timestamping method. Like Direct Measurement, this works independently of --stateful-reflector.

Timestamp Information TLV (RFC 8972 §4.3)

The Timestamp Info TLV reports the synchronization source and timestamping method at each endpoint:

stamp-suite --remote-addr 192.168.1.100 --timestamp-info

The sender fills its own sync source and method; the reflector fills in its values (e.g., NTP + software-local timestamping).

Access Report TLV (RFC 8972 §4.6)

The Access Report TLV carries an access identifier and return code. The reflector echoes it unchanged:

stamp-suite --remote-addr 192.168.1.100 --access-report 5 --access-return-code 1

Destination Node Address TLV (RFC 9503 §4)

The Destination Node Address TLV lets the sender specify the intended reflector address. The reflector checks whether the address matches any of its local interfaces:

# Verify that 192.168.1.100 is handling the reflection (requires --ssid)
stamp-suite --remote-addr 192.168.1.100 --ssid 1 --dest-node-addr 192.168.1.100

If the address does not match, the reflector sets the U-flag on the TLV and still reflects the packet, allowing the sender to detect misrouting (e.g., anycast failover).

Return Path TLV (RFC 9503 §5)

The Return Path TLV controls how the reflector routes its reply. Several sub-TLV types are supported:

# Suppress reply entirely (control code 0)
stamp-suite --remote-addr 192.168.1.100 --return-path-cc 0

# Request reply to an alternate address
stamp-suite --remote-addr 192.168.1.100 --return-address 10.0.0.5

# Request SR-MPLS return path (echoed with U-flag in userspace)
stamp-suite --remote-addr 192.168.1.100 --return-sr-mpls-labels 100,200,300

# Request SRv6 return path
stamp-suite --remote-addr 2001:db8::100 --return-srv6-sids 2001:db8::1,2001:db8::2

# Reflector: opt in to best-effort SRv6 SRH forwarding (Linux, IPv6)
stamp-suite --is-reflector --srv6-return-forwarding

The reflector handles each sub-TLV type:

  • Control Code: Bit 0 controls reply behavior (0=suppress, 1=reply); reserved bits are ignored per RFC 9503
  • Return Address: Reflector sends the reply to the specified IP. On send failure, it sets the U-flag and falls back to the original source address
  • SRv6 Segment List: Best-effort forwarding (RFC 9503 §5 + RFC 8754), opt-in via --srv6-return-forwarding. When enabled and the kernel supports it, the reflector builds an SRv6 Segment Routing Header (src/srv6.rs) and attaches it to the IPv6 reply via an IPV6_RTHDR ancillary message (sendmsg). A one-shot capability probe gates the attempt; on a non-Linux/IPv4 path, an unsupported kernel, or any send error it falls back to a normal reply with the Return Path U-flag set. Disabled by default. The SRH construction is unit-tested against RFC 8754; the live kernel send path requires an SRv6-capable testbed to exercise fully.
  • SR-MPLS: Echoed with U-flag set — forwarding an arbitrary MPLS label stack from a userspace UDP socket is out of scope (it requires raw AF_PACKET framing and next-hop resolution).

Micro-session ID TLV (RFC 9534 §3.1)

The Micro-session ID TLV enables per-member-link performance measurement within Link Aggregation Groups (LAGs). Each member link is identified by a 16-bit ID on both sender and reflector sides:

# Sender: identify this member link as ID 1
stamp-suite --remote-addr 192.168.1.100 --micro-session-id 1

# Reflector: identify this member link as ID 2
stamp-suite -i --reflector-member-link-id 2

The sender sets its member link ID in outgoing packets. The reflector validates any non-zero reflector ID in the received TLV (discards on mismatch), echoes the sender ID unchanged, and fills in its own member link ID.

Reflected Test Packet Control TLV (draft-ietf-ippm-asymmetrical-pkts)

The Reflected Test Packet Control TLV (Type 12, IANA-assigned) lets the sender request asymmetrical reply traffic — multiple reply copies spaced at a specified interval:

# Ask for 4 replies, 1 ms apart
stamp-suite --remote-addr 192.168.1.100 \
    --reflected-control-count 4 \
    --reflected-control-interval-ns 1000000

Reflector behaviour (aligned with draft-14 §3 as of this release):

  • Disabled by default (--reflected-control-max-count defaults to 0), per draft-ietf-ippm-asymmetrical-pkts, which requires the feature be administratively controllable and off by default: the reflector then sends only the single normal reply and sets the C flag (Conformant Reflected Packet, bit 3 of the TLV flags byte, mask 0x10) to signal the request was not honoured. Set a positive --reflected-control-max-count (e.g. 16) to opt in; the reflector then emits up to that many reply packets per request and clamps/​C-flags anything above it. Pair with --max-pps to bound amplification.
  • Clamps the inter-packet interval up to at least --reflected-control-min-interval-ns (default 1 µs).
  • Honours the requested reply-packet length up to --reflected-control-max-size (default 1500 bytes, typical Ethernet MTU) by appending an Extra Padding TLV (Type 1) before the HMAC TLV. When the request exceeds the cap, the C flag is set; the reply still pads to the cap.
  • Parses Layer-3 Address Group sub-TLV (sub-TLV Type 11): the reflector applies the requested prefix mask to each of its local IP addresses; if none matches, the packet is dropped per draft §3 ("MUST stop processing the received packet"). The drop surfaces to the backend as ReturnPathAction::SuppressReply.
  • Parses Layer-2 Address Group sub-TLV (sub-TLV Type 10) but cannot evaluate it on the UDP-socket backends (no MAC-address visibility). When this sub-TLV is present, the reflector sets the U flag on the echoed Type 12 TLV and continues processing the rest of the packet — the U flag signals "filter not honoured" without claiming the match passed.
  • Enforces the draft-14 §3 minimum value-field size of 12 octets at parse time. The sender path (ReflectedControlTlv::encode_value) emits 4-byte zero placeholders to satisfy this when no real sub-TLV is attached.
  • On the nix backend extra copies are sent on a spawned tokio task so the recv loop is never blocked; the pnet backend sleeps inline on its capture thread.

Bit Error Rate TLVs (draft-gandhi-ippm-stamp-ber)

Three experimental TLVs cooperate to measure residual bit errors in the Extra Padding TLV (RFC 8972 Type 1). Type numbers in the draft are TBD; this implementation uses 240/241/242 from RFC 8972's experimental range.

Type Name Direction
240 Bit Pattern in Padding sender → reflector (carries the pattern used to fill padding)
241 Bit Error Count in Padding reflector fills (u32 popcount of XOR diff)
242 Max Bit Error Burst Size reflector fills (u32 longest consecutive error run) 
# Default pattern 0xFF00, 128-byte padding
stamp-suite --remote-addr 192.168.1.100 --ber --ber-padding-size 128

# Custom pattern (hex; `0x` prefix optional)
stamp-suite --remote-addr 192.168.1.100 --ber --ber-pattern aa55 --ber-padding-size 256

The reflector XORs the received padding against the expected pattern (from the Bit Pattern TLV, or 0xFF00 if absent), counts error bits and the longest consecutive error run across byte boundaries, and writes the results into Types 241 and 242. Per draft §3, duplicate BER TLVs or a missing companion Extra Padding TLV cause the reflector to set the U-flag on all BER TLVs and skip the computation.

Reflected Fixed / IPv6 Extension Header Data TLVs (draft-ietf-ippm-stamp-ext-hdr)

Two experimental TLVs let the sender ask the reflector to echo the bytes of the received IP headers — useful for diagnosing DSCP remarking, TTL decrement, Flow Label rewriting, or tampering with IPv6 Hop-by-Hop / Destination Options by intermediate routers.

Type Name Content
246 Reflected IPv6 Extension Header Data Concatenated Hop-by-Hop (NextHeader 0) and Destination Options (NextHeader 60) extension headers, each prefixed with its NextHeader byte and HdrExtLen byte as they appeared on the wire
247 Reflected Fixed Header Data Raw 20-byte IPv4 or 40-byte IPv6 fixed header as received

Type numbers are TBD in the draft; this implementation uses 246/247 from RFC 8972's experimental range.

# Ask the reflector to reflect the IPv4/IPv6 fixed header
stamp-suite --remote-addr 192.168.1.100 --reflected-fixed-hdr

# IPv6 test with reflected extension headers
stamp-suite --remote-addr 2001:db8::1 --reflected-ipv6-ext-hdr

Backend requirement: these TLVs require the reflector to copy raw IP-header bytes into the response, which is only possible when the reflector captures at the datalink layer. Only the pnet backend can do this (see Receiver Backends).

  • On the pnet backend (Windows default, or --features ttl-pnet on Unix): the reflector populates the TLV Value with the captured bytes. For IPv4 packets the TLV is truncated to the fixed 20-byte header, so IPv4 options are not reflected.
  • On the nix backend (Linux/macOS default): the kernel hides raw IP headers from the application, so the reflector has nothing to copy. The TLVs are echoed with the sender-advertised Length preserved (zero-filled Value) and the U-flag set per RFC 8972 §4.2 and draft §3.1/§3.2 ("If, for any reason, the Session-Reflector does not use the received TLV for reflecting data, it MUST return the TLV as unrecognized"). A one-time warning tells the operator to rebuild with --features ttl-pnet if header reflection is required. The sender sees a protocol-compliant response either way.
  • A sender-requested Type 246 TLV on an IPv4 packet, or on an IPv6 packet without any extension headers, legitimately produces a zero-filled Value at the sender-advertised capacity — this is not the same as the U-flag case and is treated as a valid "no data" response.
  • Per draft-ietf-ippm-stamp-ext-hdr-08 §5.2, the Type 247 TLV Length MUST equal 20 (IPv4) or 40 (IPv6). If the sender's requested Length does not match the captured header (e.g. a 20-byte request reaches an IPv6 reflector), the reflector zero-fills the Value and sets the U-flag rather than silently truncating or zero-padding. This conformance check ships in stamp-suite as of this release.
  • Not yet implemented: the §3.1/§3.2 "non-zero first 4 bytes" disambiguation rule. When the sender pre-populates the first 4 bytes of a Type 246 TLV with header data to ask the reflector for a specific extension header (e.g. one of two same-length Hop-by-Hop options), the reflector is required to match against that pattern. Today we concatenate every captured extension header into the TLV Value regardless of the first-4-byte pattern. Tracked separately; safe today for senders that send a single TLV-instance per packet with the value field zeroed.

Prometheus Metrics

When built with --features metrics, the reflector can expose Prometheus metrics:

cargo build --release --features metrics
stamp-suite -i --metrics --metrics-addr 127.0.0.1:9090

Available metrics include:

  • stamp_reflector_packets_received_total — Total packets received
  • stamp_reflector_packets_reflected_total — Total packets reflected
  • stamp_reflector_packets_dropped_total — Dropped packets by reason
  • stamp_reflector_active_sessions — Current active sessions (stateful mode)
  • stamp_reflector_hmac_failures_total — HMAC verification failures
  • stamp_reflector_processing_seconds — Packet processing time histogram

SNMP AgentX Sub-Agent

When built with --features snmp (Unix only), stamp-suite can connect to an existing net-snmpd master agent via the AgentX protocol (RFC 2741) and expose reflector/sender state through a custom STAMP-SUITE-MIB.

cargo build --release --features snmp

# Reflector with SNMP
stamp-suite -i --snmp

# Custom AgentX socket path
stamp-suite -i --snmp --snmp-socket /var/agentx/master

# Query via net-snmp tools
snmpwalk -v2c -c public localhost .1.3.6.1.4.1.65134

The MIB (provided in mibs/STAMP-SUITE-MIB.mib) exposes:

Subtree Contents
Reflector Config Admin status, listen address/port, auth mode, TLV mode, stateful flag, session timeout
Reflector Stats Packets received/reflected/dropped, active sessions, uptime
Session Table Per-client address, port, packet counts, last sequence number, last active time
Sender Config Remote address/port, local port, packet count, send delay, auth mode
Sender Stats Packets sent/received/lost, RTT min/max/avg, jitter, loss percentage

Sender statistics are updated live during the measurement run (not just at completion), so SNMP polling reflects current progress.

Note: The snmp feature requires a Unix platform (Linux/macOS) because AgentX uses Unix domain sockets. On non-Unix platforms, --snmp prints an error and exits.

Hardware-Assisted Timestamping

Optional support for SO_TIMESTAMPING / ETHTOOL_GET_TS_INFO on Linux, gated behind the hwtstamp Cargo feature. Selected at runtime via --hwtstamp auto|on|off (default auto).

Experimental / not yet functional. Hardware and kernel timestamping are not implemented. The probe always reports "not supported", so every mode currently uses software timestamps and the Type 3 TLV always advertises SwLocal. The flag and surrounding API are scaffolding for the planned kernel work described under Status.

Defensive contract. Hardware timestamping is a per-NIC capability — some adapters support RX, some both, most consumer NICs neither. The implementation:

  • Never panics if HW support is missing.
  • Never refuses to start the binary, even with --hwtstamp on: that mode now warns that it is falling back to software rather than aborting.
  • Reports the actual method on a per-direction basis in the Type 3 Timestamp Information TLV: HwAssist only when the NIC really provided the timestamp (never, today), otherwise SwLocal.

Modes.

  • auto (default) — use HW when available, fall back silently to software. Safe on every host. (Today: always software.)
  • on — prefer HW and warn at startup when it is unavailable, so an operator who explicitly asked for HW is told they got software. (Today: always warns and uses software.)
  • off — always use software timestamps, even when HW is available. Useful for A/B comparisons or as a known-good fallback.

Capability probe. stamp_suite::hwtstamp::probe(interface) is the intended query point for kernel HW-timestamping support. It currently returns "not supported" unconditionally on every platform and build.

Status. As of this release the --hwtstamp flag, effective_method API, and Type 3 TLV reporting are in place but the feature is inert: the SCM_TIMESTAMPING cmsg read, MSG_ERRQUEUE poll, and ETHTOOL_GET_TS_INFO probe are a planned follow-up. The API is structured so the kernel-side work can land without touching call sites.

Benchmarks

benches/reflector_hotpath.rs is a Criterion harness that drives process_stamp_packet end-to-end through the in-process pipeline (no real UDP). It measures parse + HMAC + TLV processing + response assembly without the kernel scheduler in the loop — useful for catching performance regressions in the parser, HMAC code, or TLV walkers without socket-level noise.

Run:

cargo bench --bench reflector_hotpath
# or a single bench:
cargo bench --bench reflector_hotpath -- unauth_full_chain

HTML reports land under target/criterion/<bench>/report/.

Bench cases:

  • unauth_no_tlvs — 44-byte open-mode baseline.
  • unauth_one_tlv — open mode + a CoS TLV.
  • unauth_full_chain — open mode + CoS + Location + Direct Measurement
    • Follow-Up Telemetry + Timestamp Info + Access Report.
  • auth_no_tlvs — 112-byte authenticated baseline with HMAC verification on the success path.
  • auth_full_chain — authenticated mode + the same TLV chain.

Reference numbers on a 2024-era x86_64 laptop (Intel i7, single core, release build): unauth_no_tlvs ≈ 100 ns/op (~10 Mpps single-threaded parse-and-assemble); auth_no_tlvs ≈ 1.5–2 µs/op dominated by HMAC. Real numbers vary with CPU, OpenSSL/RustCrypto build, and tokio runtime overhead in the receive path. Treat the benches as a regression signal, not as headline marketing figures.

See Also

  • README — install and quick-start.
  • usage.md — configuration file format, full CLI flag reference.
  • security.md — HMAC, key management, systemd hardening, capability model.