observability: implement structured logger and EVENTS registry

[AbdelStark]

Problem

RFC-0013 / docs/spec/05-observability.md define a JSONL structured-log contract that every other subsystem references (training metrics, eval book-ends, cache hit/miss, attestation verify, error events). Nothing downstream can take a hard dependency on event names until the EVENTS registry and the per-run JSONL sink land.

Solution

• geno_lewm/observability.py

  • EVENTS: tuple of EventSpec(name, severity, summary) covering all 22 canonical v0.1 events. Renaming a name is a MAJOR change. Duplicate-detection at import.
  • LogRecord: dataclass carrying every spec-required field; standardized optional fields (step, epoch, phase, duration_ms, trace_id, span_id, error_code) are emitted only when non-None.
  • get_logger(component, run_id?, log_dir?, level?, pretty?): cached factory; identical args return the same instance, so independent subsystems share a single ordered stream per run.
  • JSONL sink: line-buffered, append-only, lock-serialized. Default path ${GENO_LEWM_LOG_DIR}/{run_id}.jsonl, falling back to ~/.geno-lewm/logs/. Concurrent threads safe.
  • Pretty stderr: auto-on when stderr is a TTY (or GENO_LEWM_LOG_FORMAT=pretty); jsonl otherwise.
  • Trace context: contextvar pair + set_trace_context() block; IDs attach only when set.
  • logged_run(): context manager that flushes the sink on any exception. For GenoLeWMError subclasses, emits a final event="error" record carrying the typed error_code before re-raising (INV-OBS-6).

• tests/unit/test_observability.py (20 cases):

  • Canonical event coverage (locks the spec list).
  • Required-field record shape; ISO-8601-ms Z timestamp; severity threshold; standardized-field promotion; trace context present/absent; error_code only when supplied; factory caching; env-var log-dir resolution; unique default run_id; JSONL path; book-end events; crash survival (typed + untyped exceptions); set_level rejection raises InputError; thread-safety smoke (4 threads × 50 writes); JSON isolation of data.

Validation

$ python -m pytest tests/ -q
164 passed in 0.14s

$ python -m tools.lint.check_error_codes
$ echo $?
0

The error linter from #22 immediately caught two RuntimeError / ValueError raises in this module's first draft. Both replaced with InvariantViolation / InputError (the public surface intentionally takes the typed-error contract — see RFC-0012).

Caveats / out of scope (deferred to follow-ups)

• Redaction filter (observability: implement redaction filter (default strict) #24): the logger currently accepts whatever payload it is given. The filter will plug into GenoLeWMLogger._emit between record construction and JSON serialization.
• Metrics registry & Prometheus exporter (observability: implement metrics registry + Prometheus textfile exporter #25): separate issue.
• wandb sink (observability: opt-in wandb sink for training metrics #26) / OTel OTLP exporter: opt-in, separate issues.
• registered_event_name AST linter (observability: AST linter registered_event_name and registered_metric_name #27): at runtime unknown event names are still emitted (no crash); the linter is what closes INV-OBS-1.
• Sampling: spec'd in §"Sampling and rate limits". The hook lives in GenoLeWMLogger._log (level threshold today) and will extend when training paths use it.

Closes #23