Skip to content

observability: AST linter check_event_names#108

Merged
AbdelStark merged 1 commit into
mainfrom
observability/ast-linter-events
May 20, 2026
Merged

observability: AST linter check_event_names#108
AbdelStark merged 1 commit into
mainfrom
observability/ast-linter-events

Conversation

@AbdelStark

Copy link
Copy Markdown
Owner

Problem

RFC-0013 INV-OBS-1/2 require every logged event name and every metric name to be a literal registered in EVENTS / METRICS. Without a linter, a typo like logger.info('training.step') vs 'training_step' slips through silently.

Solution

  • tools/lint/check_event_names.py

    • registered_event_name: walks every logger.{debug,info,warn,warning,error}(...) call. If the first positional arg is a string literal, asserts it is in EVENTS. Dynamic event names (Name or f-string) are skipped — they're a runtime check.
    • registered_metric_name: same shape for counter.inc() / histogram.observe() / gauge.set(). Receiver-name heuristic restricts the check to obvious metric handles to avoid flagging unrelated .inc() calls.
    • Discovers both registries by AST-parsing observability.py — no runtime import.
    • Skips observability.py itself (the EventSpec(...) literals would otherwise look like raises).
    • Returns early on METRICS discovery if absent, so this check stays dormant until observability: implement metrics registry + Prometheus textfile exporter #25 ships.
  • .github/workflows/lint-errors.yml renamed to lint and gains a check-event-names job alongside check-error-codes.

  • tests/lint/test_check_event_names.py (19 cases): registry discovery; positive / negative event names; dynamic / f-string event names skipped; parametrized over all logger levels; metric check armed only when registry present; format spec; self-skip; no-args tolerated.

Validation

$ python -m pytest tests/ -q
224 passed in 0.69s

$ python -m tools.lint.check_event_names
$ echo $?
0

Caveats

  • METRICS-check is structurally implemented but stays dormant until observability: implement metrics registry + Prometheus textfile exporter #25 lands the registry. When it does, no linter change is needed.
  • False-positive rate on the metric receiver heuristic was the open trade-off. Decided to require a counter|histogram|gauge|metric substring in the receiver identifier — strict enough that ordinary box.inc(1) does not trigger.

Closes #27

Add tools/lint/check_event_names.py implementing two AST checks
required by RFC-0013 INV-OBS-1/2 and RFC-0015 §3.4:

- registered_event_name: every logger.{debug,info,warn,warning,error}
  call site that passes a string literal as the first positional
  argument uses a name registered in EVENTS.
- registered_metric_name: every counter.inc/observe/set call site
  with a string-literal first argument uses a name registered in
  METRICS. The check stays dormant until METRICS ships (#25) by
  returning early when discover_registered_metrics() returns None.

The metric-call detector is intentionally name-heuristic to avoid
flagging vector .inc()/.set() math. It only triggers when the receiver
identifier contains counter/histogram/gauge/metric.

Parses observability.py with ast to discover the legal name set so
the linter has no runtime dependency on the package.

Tests in tests/lint/test_check_event_names.py cover:
- registry discovery against the real module
- registered/unregistered events (positive + negative)
- dynamic event names (variable / f-string) skipped
- every logger level method (debug/info/warn/warning/error)
- metric check armed only when METRICS is present
- unrelated .inc() calls ignored
- self-skip on observability.py
- format spec, main() exit codes, no-args call tolerated

The .github/workflows/lint-errors.yml workflow renames to ``lint``
and gains a check-event-names job alongside check-error-codes.

Closes #27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

observability: AST linter registered_event_name and registered_metric_name

1 participant