Surface inlet connection status and re-resolve lost streams

[cboulay]

Two commits. The first makes inlet status transitions visible; the second acts on the failure the first one exposed.

1. Log inlet connection status transitions

An inlet that never resolves, one that resolved the wrong stream, and one attached to a stream that is simply quiet are indistinguishable from outside — all three are a unit that emits nothing. Three log lines, one per transition (connected / first failed resolve / pull raised), each emitted once per connection since all three conditions are polled every tick.

2. Re-resolve lost streams instead of relying on liblsl

liblsl's built-in recovery re-acquires by source_id alone. That is wrong for us twice over:

• It ignores the host criterion. _try_connect honors info.host; liblsl's recovery does not, and its docs are explicit that a stream may return "even after the serving app, device or computer crashes". A same-named stream on another machine can be silently substituted.
• It never gives up, and never raises. An upstream that returns with a different shape has a different source_id, so it is never re-acquired and the inlet stalls forever — looking exactly like a quiet stream.

So: recover=False, and reconnect here.

This costs no data

Measured, not assumed:

Scenario	Result
Destroy inlet, re-resolve while outlet lives (50 buffered + 11 pushed while detached)	0 recovered
Outlet dies with 50 samples in the inlet's receive buffer	samples_available() = 51, pull_chunk raises LostError, 0 drained

The outlet fans out to a per-consumer queue that exists only while that consumer is attached, and there is no resume-by-offset in the LSL protocol — the 11 samples pushed while nothing was connected were lost too. No backlog survives for either mechanism to reclaim. And liblsl checks the lost state before draining, so the inlet's own buffer is unreachable once the stream is declared lost — a "drain before teardown" salvage path is not possible. Loss is already unavoidable today; this only makes it visible.

If the blip is short enough that TCP survives, nothing raises and no reconnect happens — that path is unchanged.

Changes

• LostError is flagged on the pull worker and acted on in _produce, which calls _request_reset(). _reset_state already drops the connection and rebuilds the ContinuousResolver, so that is the entire teardown. The old inlet is released rather than closed, so an in-flight pull holding it via its snapshot stays valid.
• Within reconnect_grace_dur (5 s) only the original source_id is accepted; afterwards any criteria match is. The host filter applies throughout.
• uid, source_id, hostname stamped onto every message via attrs. uid identifies the outlet instance and so distinguishes a dropped socket from a restart; source_id is deliberately stable across restarts and cannot.
• distinct_key_per_connection (default off) folds a uid-change counter into key. Downstream hashes on (shape, rate, key) — verified in window.py:254, downsample.py:54 — so a restarted upstream that keeps its name and shape otherwise carries stale filter state and half-full windows across the gap. Off by default because key names NWB containers (writer.py:257) and routes pipelines.
• WARNING when a reconnect lands on a different shape or host, naming the delta.
• Bounded the previously unbounded info() fetch; a connect that opens but cannot fetch its description is now retried rather than hanging.

Tests

43 pass. The end-to-end test drives a real outlet, kills it, and brings up a replacement with a different source_id and channel count — the case liblsl would never re-acquire. Confirmed meaningful: with recover=True it fails at "a vanished outlet never surfaced as a lost stream".

🤖 Generated with Claude Code