Skip to content

V8.3-SPEC.md

Latest commit

 

History

History
322 lines (260 loc) · 17.5 KB

V8.3-SPEC.md

File metadata and controls

322 lines (260 loc) · 17.5 KB
Raw

V8.3-SPEC — LoRaWAN (sync 0x34) keyless bridge/relay/mesh + v8.2.1 carry-overs

Status: APPROVED + IMPLEMENTED (build-green) — awaiting owner bench. LoRaWAN design approved 2026-06-13 (LW-Q1…Q5 answered, §11). All of §3, §4, §5, §6, §7 and the §8 portal seam are implemented on branch v8.3-dev, each commit pio run -e xiao_esp32s3 green. Bench (§9) is the only remaining gate before tag v8.3. Deferred ABP/OTAA decode/encode (§10) is NOT in v8.3.

  • Branch: v8.3-dev off main @ 9723309 (tag v8.2.1).
  • Working clone: C:\Users\6r4yh\workspace\Platformio\Projects\Xiao-esp32s3-lora-repeater - main dev-ABP-lorawan.
  • Process gate (unchanged): spec approval → implement (one logical change per commit, pio run -e xiao_esp32s3 green each) → owner bench → annotated tag v8.3 + push. Nothing is pushed during implementation. Never force-push main.

1. Scope (owner decisions locked 2026-06-13)

v8.3 delivers, in this order:

  1. (DONE) Learn wall-clock from Meshtastic POSITION_APP — the outstanding v8.2.1 follow-up. §3.
  2. (DONE) Transparent in-protocol Reticulum repeat (RNS → RNS) — the audit fix that brings RNS in-protocol bridging in line with the v8.2 routing. §4.
  3. LoRaWAN (LoRa sync 0x34) — keyless feature set:
    • Transparent LW ↔ LW raw relay — a localized transparent bridge modeled after the Tasmota LoRa & LoRaWAN Bridge. §6.
    • Metadata capture tap (cleartext header decode/log/summary). §5.
    • Multi-hop raw flood "mesh" (dedup-bounded flood across ≥2 bridges). §7.

Explicitly DEFERRED to v8.3+ (the carry-over task list, §10): ABP/OTAA key-based decode of the operator's own devices (→ MT/MC/custom) and encode (MT/MC → LoRaWAN). No cryptographic key material, no FRMPayload decrypt/encrypt, no FCnt/MIC synthesis in v8.3.

Why keyless is the whole v8.3 LoRaWAN story: V8.2-SPEC §14 established (verified against LoRaWAN L2 1.0.x/1.1 + Semtech) that LoRaWAN content bridging is impossible without per-device session keys. The owner has chosen to keep keys out of v8.3, so v8.3 operates only on the cleartext PHY frame: read the unencrypted header (capture tap) and re-transmit the frame byte-for-byte (relay/mesh). Both are sound and need no keys. The key-based "decode my own fleet" path that §14 left open is real and feasible, but it is v8.3+ work.

2. What v8.3 reuses from v8.2 (no new routing machinery)

The keyless LoRaWAN feature is structurally identical to the RNS → RNS repeat shipped in §4 — it adds a protocol, not a routing model:

v8.2/v8.3 primitive Role for LoRaWAN
DedupCache (hash of raw frame) Loop suppression for relay/flood; one repeat per unique frame per bridge
rawRepeatForDest() Byte-for-byte re-transmit of the 0x34 PHYPayload to another 0x34 radio
RouteQueue + CAD-gated non-blocking TX + airtime throttle Paces relayed frames; a relay never blocks the other radio's RX
sync-word-driven dispatch in ingestAndFanout New 0x34 source case slots in beside MC/MT/RNS
enqueueTextForDest dest switch New 0x34 dest case = log-and-drop for MT/MC→LW (no keyless inject)
resolveRadioChannel else-branch (no key) 0x34 needs no channel key — already handled

3. (DONE) Learn wall-clock from Meshtastic POSITION_APP

Problem. The bridge has no RTC/NTP. v8.2.1 taught it to learn wall-clock from inbound MeshCore timestamps, but a fresh boot stamps MT→MC packets at 1969 until the first timestamped MeshCore packet arrives.

Fix (commit 9499804).

  • MeshDecoderDebug.h: MeshtasticPositionInfo gains hasTime/timeUnix. extractMeshtasticPosition now parses Position.time (field 4, fixed32, Unix seconds — authoritative) and falls back to Position.timestamp (field 7, fixed32, the GPS-solution time) when field 4 is absent. Field 4 is noted in the protobuf as "potentially unused for mesh transmission," so field 7 materially raises the calibration hit-rate. hasTime is not part of the return gate, so a coordinate-less position is still not bridged. (Field numbers verified against meshtastic/protobufs mesh.proto.)
  • main.cpp: the MeshCore-only clock helpers are generalized into learnClock(ts, src) with learnClockFromMc/learnClockFromMt wrappers (g_mcClock* → g_clock*). The POSITION decode path calls learnClockFromMt when a coord-bearing position carries a plausible (> 2017) time.

Coupling note (for bench/review): clock-learn fires inside the positionEnabled()-gated decode, so disabling position bridging also disables this calibration source (MeshCore calibration is unaffected). Decoupling is a one-line future option, not in v8.3.

Bench: boot with no MC traffic, inject an MT POSITION_APP carrying a real time → expect evt=CLOCK src=MT unix=… once, and subsequent MT→MC packets stamped with a real mcts= instead of 0/1969.

4. (DONE) Transparent in-protocol Reticulum repeat (RNS → RNS)

Audit finding (your item #4). The bridge had three RNS directions; only the in-protocol one was broken:

Direction Pre-v8.3 status
RNS → MT/MC works — base64 fragment tunnel ([rns AA X/Y]), paced by v8.2 RouteQueue/CAD
MT/MC → RNS log-and-drop no-rns-encoder (no RNS encoder; v8.3+ item)
RNS → RNS silently skipped — enqueueReticulumForDest returned early for an RNS destination

So a bridge with both radios on Reticulum could not act as a transparent in-protocol repeater: the v8.2 same-channel raw-repeat (rawRepeatForDest) was added for MT/MC but never extended to the RNS source path.

Fix (commit e337afd). In the RNS source fan-out, an in-protocol RNS destination now goes through rawRepeatForDest (PHYPayload re-transmitted byte-for-byte; RNS Transport on the end nodes handles hop limits and network-level dedup), paced by the v8.2 RouteQueue + CAD + airtime throttle. Loop-safe with no extra dedup record — the raw-frame hash is already recorded at ingest, so our own echo is dropped on the way back in. Gated by BRIDGE_RNS_INPROTO_REPEAT (default 1); 0 restores tunnel-only behaviour. RNS → MT/MC and MT/MC → RNS are unchanged.

Bench: two radios both on 0x42 at different frequencies; send an RNS frame into radio A → expect evt=QUEUE … mode=raw to radio B and the byte-identical frame on air; confirm the echo back into A is dropped (drop=rns-dup).

5. LoRaWAN — metadata capture tap (keyless)

A one-way situational-awareness tap, the honest scope from V8.2-SPEC §14.2.

5.1 Cleartext frame layout (what is parseable without keys). LoRaWAN PHYPayload = MHDR(1) | MACPayload | MIC(4).

  • MHDR: MType = bits[7:5], Major = bits[1:0]. MType values: 0 JoinRequest, 1 JoinAccept (encrypted), 2 UnconfirmedDataUp, 3 UnconfirmedDataDown, 4 ConfirmedDataUp, 5 ConfirmedDataDown, 6 RejoinRequest, 7 Proprietary.
  • Data frames (MType 2–5): MACPayload = FHDR | [FPort] | [FRMPayload], FHDR = DevAddr(4, LE) | FCtrl(1) | FCnt(2, LE) | FOpts(0–15), FCtrl low nibble = FOptsLen. FPort present iff payload remains after FOpts. FRMPayload is AES-encrypted (AppSKey) — never decoded in v8.3.
  • JoinRequest (MType 0): JoinEUI(8, LE) | DevEUI(8, LE) | DevNonce(2, LE) — all cleartext.
  • Cleartext, learnable fields: MType, DevAddr, FCtrl flags, FCnt, FPort, payload length, and (1.0.x) FOpts MAC commands; plus PHY metadata (RSSI/SNR/SF/freq). DevAddr is locally assigned (changes on re-join), not a stable hardware id.

5.2 Implementation.

  • MeshDecoderDebug.h: SYNC_WORD_LORAWAN = 0x34 + printLoRaWAN() (header hex/field dump, matching the existing print* debug style) + a small extractLoRaWANMeta(buf, len, out) returning a struct { mtype, devAddr, fcnt, fport, hasFport, fctrl, payloadLen }.
  • main.cpp ingestAndFanout: new 0x34 source case — dedup the raw frame (hash(buf,len,0), like RNS), then:
    • emit a structured evt=RX proto=LW devaddr=0x… mtype=… fcnt=… fport=… len=… rssi=… snr=… sf=… log line (§13 schema, new proto=LW), and
    • optionally emit a one-line MC/MT summary ("LoRaWAN DevAddr 0x26011F8A FCnt 4213 FPort 2 SF7 -91dBm") gated by a config toggle (see LW-Q2). This is metadata, never message content.

6. LoRaWAN — transparent LW ↔ LW raw relay (keyless)

The Tasmota "localized transparent bridge" model: extend coverage by repeating 0x34 frames between two radios (e.g. two sub-bands, or a far device → a gateway's band).

6.1 Mechanism. In the 0x34 source case, for each other enabled radio that is also 0x34, call rawRepeatForDest(...) — the same primitive used for RNS → RNS in §4. The PHYPayload (incl. MIC) is re-transmitted byte-for-byte; we never touch FCnt/MIC (we are a dumb L2 repeater, not a network server).

6.2 Why this is LoRaWAN-correct. A real LoRaWAN deployment already expects the same uplink to be received by multiple gateways ("gateway diversity"); the Network Server dedups duplicate uplinks by DevAddr+FCnt+ MIC. A transparent relay simply makes our radio one more receive path to the NS — byte-identical frames are exactly what the NS already tolerates. We do not forge frames, so no MIC/FCnt problem arises.

6.3 Loop-safety. DedupCache records hash(raw frame) at ingest, so a frame is repeated at most once per bridge and our own echo is dropped. No frame mutation, so the hash is stable across the repeat.

6.4 Honest limitations (document in README).

  • Single-channel per radio. Each radio listens on one freq/SF; LoRaWAN channel-hopping (e.g. US915's 64+8 channels) is not followed. We capture/relay only what each radio is tuned to — identical to a single-channel LoRa bridge. (LW-Q4.)
  • No RX-window timing. We relay opportunistically via the airtime throttle; we do not honour Class-A RX1/RX2 downlink windows. Fine for uplink coverage extension; not a substitute for a real gateway/relay appliance.

7. LoRaWAN — multi-hop raw flood "mesh" (keyless)

Definition (owner): multi-hop raw flood relay. With ≥2 bridges deployed, each bridge floods a unique 0x34 frame outward exactly once; the shared DedupCache (TTL-bounded) is the loop/flood bound. This is §6 generalized across more than two radios/bridges — no new code path beyond "repeat to every other enabled 0x34 radio, dedup-gated."

Key design constraint — no embeddable hop counter. A LoRaWAN frame carries a MIC; mutating any byte (to decrement a TTL) invalidates the frame. So a transparent flood cannot carry an in-band hop count without breaking the frame for every other LoRaWAN node. Flood bounding therefore relies on:

  • dedup (each bridge repeats each frame once), and
  • the DedupCache TTL (bounds how long a frame can circulate), and
  • the airtime throttle (bounds channel occupancy).

This is the same mechanism v8.2 uses for MC/MT mesh-flood suppression — honest and correct, but it means "mesh" here = dedup-bounded transparent flood, not a TTL-routed mesh. This must be stated plainly in the README so operators don't expect hop-limited routing. (LW-Q3.)

8. The seam — exact files (verified against the post-v8.2 baseline)

File Change Notes
MeshDecoderDebug.h SYNC_WORD_LORAWAN = 0x34; printLoRaWAN(); extractLoRaWANMeta() header-only, isolated
BridgeConfig.h PROTO_LORAWAN = 5 enum value (LW-Q1) portal protocol selection
BridgeConfig.cpp PROTO_LORAWAN → syncWord 0x34; LoRaWAN RF defaults per region sync + RF plan
main.cpp ingestAndFanout 0x34 source case: dedup → capture-tap log/summary → raw-repeat to other 0x34 radios mirrors the RNS source branch from §4
main.cpp enqueueTextForDest 0x34 dest case: log-and-drop (MT/MC→LW has no keyless inject) mirrors the RNS no-encoder drop
main.cpp resolveRadioChannel none — 0x34 falls to the no-key else-branch already handled
CaptivePortal.cpp "LoRaWAN" protocol option + RF defaults + capture/relay toggles ~portal form
LoraConfigCheck.h none (sync is range-checked, accepts 0x34) 0
platformio.ini document BRIDGE_LW_* build-flag defaults knob docs

SX126x sync-word note (implementation detail to verify): 0x34 is the LoRaWAN public sync word on SX127x; the SX126x programmed equivalent is 0x3444. Confirm WioSX1262's sync-word path maps the logical 0x34 to the correct SX126x register value (RadioLib setSyncWord) at implementation time.

Build flags (as implemented):

  • BRIDGE_LW_CAPTURE (default 1) — emit the evt=RX proto=LW capture log.
  • BRIDGE_LW_SUMMARY_TO_MESH (default 1, per LW-Q2) — also emit a metadata summary line to each MT/MC destination. Set 0 to keep LoRaWAN metadata on the serial log only (no added mesh traffic).
  • BRIDGE_LW_RELAY (default 1) — transparent LW↔LW raw repeat (the §6/§7 core).

9. Bench plan (owner-gated, before tag v8.3)

The full executable protocol is in BENCH-v8.3.md — equipment + stimulus generators, captive-portal config profiles, canonical hand-built frame vectors, an ordered 27-test matrix, per-test exact expected serial lines + pass/fail, and a completeness check. The summary below is the overview; BENCH-v8.3.md is authoritative.

No real LoRaWAN infrastructure is required; a second SX1262 (or another Xiao radio) programmed to sync 0x34 on the same freq/SF can generate test frames.

  1. Capture tap — inject a known UnconfirmedDataUp frame → expect a correct evt=RX proto=LW devaddr=… fcnt=… fport=… decode (cross-check the hand-built frame's header bytes).
  2. JoinRequest decode — inject an MType-0 frame → expect mtype=join with DevEUI/JoinEUI surfaced; no payload decode attempted.
  3. Transparent relay — two radios on 0x34/different freq; frame into A → byte-identical frame out of B (mode=raw); confirm A drops the echo.
  4. Flood dedup — re-inject the same frame → drop=…dup; confirm one repeat only.
  5. MT/MC → LW drop — confirm a text packet destined to the LW radio log-and-drops (drop=no-lw-encoder), no malformed 0x34 emission.
  6. Regression — re-run the §3/§4 carry-over checks and a v8.2 MC/MT/RNS regression pass so the new source case doesn't perturb existing routing.

10. Deferred to v8.3+ (the carry-over task list)

Recorded here, in CLAUDE.md's roadmap, and in cross-session memory — not the harness task tracker (per CLAUDE.md project rule).

  1. LoRaWAN key-based decode (my fleet → MT/MC/custom). With operator-provisioned keys, decrypt FRMPayload for the operator's own devices and emit as MC/MT text or a custom setting. The §14 "decrypt my own fleet" path.
    • ABP first (static DevAddr+NwkSKey+AppSKey; no Join state machine), then OTAA (DevEUI+JoinEUI+AppKey; bridge observes/participates in the Join to derive session keys; requires FCnt persistence in NVS).
  2. LoRaWAN encode (MT/MC → LoRaWAN). Inject text into the operator's own LoRaWAN devices/network. Hardest direction — needs DevAddr+keys+monotonic FCnt+MIC+RX-window timing; only works for fully owned devices.
  3. Reticulum bidirectional encode (MT/MC → RNS). The RNS encoder + fragment reassembly that turns the current one-way tunnel into a full bridge.
  4. (carry-overs from v8.2) MeshCore identical-text dedup; §15.2 decoder-dump log-mutex; Sub-GHz ↔ 2.4 GHz cross-band (LR1121, Seeed-blocked).

11. Owner decisions (RESOLVED 2026-06-13)

  • LW-Q1 — protocol enum → dedicated PROTO_LORAWAN = 5. ✅ Done: the portal picker pre-seeds sync 0x34; protocolFromSync(0x34) → PROTO_LORAWAN.
  • LW-Q2 — capture-tap output → log AND metadata summary. ✅ Done: BRIDGE_LW_CAPTURE logs evt=RX proto=LW; BRIDGE_LW_SUMMARY_TO_MESH defaults 1 and emits a one-line summary to each MT/MC destination.
  • LW-Q3 — "mesh" semantics → dedup/TTL-bounded flood, confirmed. ✅ No in-band hop counter (would break the MIC); flood is bounded by the shared DedupCache + airtime throttle. To be documented in the README.
  • LW-Q4 — single-channel → accepted. ✅ One freq/SF per radio; no LoRaWAN channel hopping (Tasmota-equivalent limitation).
  • LW-Q5 — relay frame scope → relay ALL 0x34 frame types raw. ✅ rawRepeatForDest is type-agnostic; the capture tap decodes data + join-request headers, opaque types (JoinAccept/Rejoin/Proprietary) report mtype only.

12. References

  • LoRaWAN L2 1.0.4 / 1.1 specification (LoRa Alliance) — frame format (MHDR/FHDR/FCtrl/FCnt/FPort/MIC), MType table, AES-CTR FRMPayload, AES-CMAC MIC, gateway-diversity dedup at the Network Server.
  • Semtech SX1262 datasheet / RadioLib SX126x — LoRa sync-word programming (0x34 public → 0x3444 on SX126x).
  • Tasmota — LoRa & LoRaWAN Bridge — https://tasmota.github.io/docs/LoRa-and-LoRaWan-Bridge/ (the transparent/localized bridge model this feature mirrors; key-based decode of one's own devices is their LoRaWAN mode, deferred here to v8.3+).
  • Meshtastic protobufs meshtastic/mesh.proto — Position.time (field 4) / .timestamp (field 7), used by §3.
  • V8.2-SPEC.md §14 — the original LoRaWAN feasibility analysis this spec extends from "capture-only" to "keyless capture + transparent relay/mesh."