Commit e10e140
authored
Log lock-release lifecycle in executor_step_status and distributed_lock (#5027)
## Summary
Adds two INFO-level log lines so the next occurrence of the
distributed-lock self-race described in #5026 can
be identified from logs alone.
- `StatusFile.write_status`
(lib/marin/src/marin/execution/executor_step_status.py) logs `Releasing
lock path=... worker=... reason=terminal_status:<STATUS>` before the
conditional `release_lock()` branch on terminal statuses.
- `DistributedLease.release`
(lib/rigging/src/rigging/distributed_lock.py) logs `Released lock
path=... worker=...` at INFO immediately after `self._delete()` (the
prior DEBUG line in the same spot was promoted to INFO rather than
duplicated).
Together these disambiguate a self-release from an external delete or a
stale-lease takeover — the existing `LeaseLostError` message at
distributed_lock.py:152 cannot tell them apart today.
Implements diffs #1 and #2 from the issue's "Instrumentation gap"
proposal. Diff #3 (the `refresh` error-message fix) is not included here
and can follow up if/when useful.
Refs #5026
## Test plan
- `./infra/pre-commit.py --all-files --fix` — passes (ruff, black,
pyrefly, license headers, etc.)
- `uv run pytest lib/rigging/tests -m "not slow"` — 66 passed
- `uv run pytest lib/iris/tests/test_distributed_lock.py -m "not slow"`
— 16 passed (covers the modified `DistributedLease.release` path)
Reviewers: please confirm the log-line wording is what you want to see
in prod, and that promoting the existing `[%s] Released lock %s` DEBUG
line to the new INFO shape (rather than keeping both) is the right call.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Romain Yon <1596570+yonromai@users.noreply.github.com>1 parent 154ec05 commit e10e140
2 files changed
Lines changed: 7 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
123 | 129 | | |
124 | 130 | | |
125 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
164 | | - | |
| 164 | + | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
| |||
0 commit comments