Skip to content

build(tooling): run the hooks with prek and manage Python deps with uv - #350

Merged
Minipada merged 1 commit into
jazzyfrom
chore/prek-uv-migration
Aug 17, 2026
Merged

build(tooling): run the hooks with prek and manage Python deps with uv#350
Minipada merged 1 commit into
jazzyfrom
chore/prek-uv-migration

Conversation

@Minipada

@Minipada Minipada commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Mirrors the move already made in ~/dev/monorepo: .pre-commit-config.yaml is now run by prek — same file, same schema, one Rust binary that resolves hook environments with uv and needs no Python environment of its own — and dependencies live in a PEP 621 pyproject.toml + uv.lock instead of poetry's [tool.poetry] + poetry.lock.

No issue for this one; it came out of the observation that the formatting gate had stopped running at all.

The formatting job was dead, not slow

.github/workflows/format.yaml installed poetry and then pip-installed requirements.txt + requirements-dev.txt just to get pre-commit on PATH. That install had been failing outright — ERROR: Failed to build 'fastcov' … ModuleNotFoundError: No module named 'pkg_resources' — so its last two runs never reached a single hook, and the workflow was then disabled manually. Every finding below is therefore pre-existing drift the dead gate was hiding, not new. (The workflow is enabled again as of this PR, which is where it is first exercised: 24 hooks, 25 seconds.)

The job now installs prek and nothing else: no poetry, no pip, no apt. clang-format arrives in the mirror hook's own wheel, check-xml uses Python's parser rather than xmllint, and graphviz was only ever there for pydeps, which nothing calls.

Python lint/format is ruff only

flake8 + 13 plugins, black, isort, pycln, pyupgrade, flynt, yesqa and the python-no-log-warn pygrep hook are replaced by ruff-check + ruff-format — one binary instead of ~20 hook environments pre-commit had to build before linting a line. [tool.ruff.lint]'s selection is chosen for parity with what the old stack enforced (E/W/F, I, UP, B, C4, A, Q, N, G/LOG, T10, EXE, and RUF100 for yesqa), not for maximum strictness: ruff's newer opinions (the rest of RUF, SIM, PTH, …) are deliberately left off, the same "broaden over time" stance the monorepo's first rollout took. setup.cfg's [flake8]/[darglint] sections are gone; darglint has no ruff equivalent worth turning on blind and is not replaced.

Fallout, all mechanical, fixed rather than ignored: 16 findings from the parity rule set (unsorted imports, a stale # noqa: T201 ×3 for a rule this repo never enabled, datetime.utcnow, three (str, Enum) classes → StrEnum, two bare zip()s → strict=True, three root-logger calls in the dashboard → a module logger, one dict comprehension) and 14 files reformatted by ruff-format (mostly black 23's missing blank line after a module docstring). Two more the dead job had hidden: dc_triggers' two .cpp files were never clang-formatted, and tools/e2e/Containerfile's find … | xargs tripped hadolint DL4006/SC2038 — rewritten as find … -exec … +, which drops the pipe instead of silencing the rule.

Dependencies

requires-python moves from the humble-era >=3.8.10,<3.9.7 || >3.9.7 to >=3.12 (Jazzy/Ubuntu 24.04), with .python-version pinning 3.12 so a local uv sync builds against the interpreter the code actually runs on. Deliberately held below 2: numpy (Jazzy's rclpy/cv_bridge are built against 1.26) and pydantic (draw_image.py and the dashboard still use v1 APIs — BaseSettings).

Dropped from the dev group as used by nothing: ipython, pydeps, pylint, cmake-format, plus strictdoc and fastcov, which live in the containers that actually run them (containers/doc/Containerfile, tools/e2e/Containerfile) — fastcov's sdist is what broke CI in the first place.

requirements.txt and requirements-dev.txt are deleted, not re-exported: pyproject.toml + uv.lock are the single dependency source, and a generated copy in the tree is only somewhere for the two to disagree. doc/src/dc/setup.md and the two AWS demo pages now say uv sync (--no-dev for the runtime set alone), and .dockerignore allows pyproject.toml + uv.lock in their place.

The ROS manifest hooks are vendored

minipada/ros2_pre-commit's ros2_ws_same_versions finds setup.py with an unbounded find "$SRC" -name setup.py, so once uv sync puts a virtualenv at ./.venv it checks every setup.py in site-packages — numpy's, for one — against the DC version. Both its hooks also shell out to xmllint, which GitHub's runner image doesn't ship, and that was the only system dependency left in the whole hook set.

tools/ci/pre-commit/ros2_package_checks.py does both checks over dc_*/ only (versions: one version across every package.xml and each dc_*/setup.py declaring it; metadata: no package left the template's placeholder license, maintainer or description), reading manifests with ElementTree — the same parser the check-xml hook already uses. stdlib rather than defusedxml: the input is this repo's own manifests, and the dependency would mean giving the hook a Python environment it otherwise doesn't need.

Also: a comments rule in CLAUDE.md

File headers one to three lines, block comments one or two, why not what, long rationale in progress.txt / an ADR / the PR description. Prompted by this branch's first draft of format.yaml, which carried a 22-line header essay. ci.yaml, doc.yaml and tools/e2e/Containerfile predate the rule — to be trimmed when next touched, not copied.

Verification

  • CI: Formatting (prek) passes on this PR — all 24 hooks in 25s, hadolint's docker-image hook included.
  • prek run --all-files --skip build-doc passes locally too (adds build-doc, which doc.yaml owns in CI).
  • uv sync --frozen resolves and installs on 3.12; every dev-group and runtime package imports, and ruff from that venv is the same 0.16.3 the hook pins.
  • All 63 tracked .py files byte-compile. check_if_numbers_are_consecutive and the dashboard's three StrEnums were smoke-tested directly (.value, == against a plain str, and iteration order all unchanged).
  • The vendored checks were negative-tested against a scratch workspace at the same depth: a mismatched package.xml version, a mismatched setup.py version and a package with all three placeholder fields are each reported, and both checks pass once the workspace agrees (single-quoted version='…' included, which upstream's grep did not accept).

🤖 Generated with Claude Code

https://claude.ai/code/session_015aZ8Ng8bK4mK9keCKxPzwX

@Minipada
Minipada force-pushed the chore/prek-uv-migration branch 3 times, most recently from 9195be0 to 95ea6b1 Compare August 17, 2026 18:07
Mirrors the move already made in ~/dev/monorepo. .pre-commit-config.yaml is now
run by prek -- same file, same schema, one Rust binary that resolves hook
environments with uv and needs no Python environment of its own -- and
dependencies live in a PEP 621 pyproject.toml + uv.lock instead of poetry.

The formatting job was dead, not slow: it installed poetry and then pip-installed
requirements.txt + requirements-dev.txt just to get pre-commit on PATH, and that
install has been failing outright ("Failed to build 'fastcov' ... No module named
'pkg_resources'"), so the last two runs never reached a hook. It now installs
prek and nothing else -- no apt step either.

Python lint/format becomes ruff only: flake8 + 13 plugins, black, isort, pycln,
pyupgrade, flynt, yesqa and python-no-log-warn are gone, and [tool.ruff.lint]
selects for parity with what they enforced rather than for maximum strictness.
Fixing the fallout is most of this diff -- 16 lint findings and 14 files
reformatted, plus two failures the broken job had been hiding: dc_triggers' two
.cpp files were never clang-formatted, and tools/e2e/Containerfile's
`find | xargs` tripped hadolint DL4006/SC2038.

requires-python moves to >=3.12 (Jazzy), with numpy and pydantic deliberately
held below 2 -- rclpy/cv_bridge are built against numpy 1.26, and draw_image.py
and the dashboard still use pydantic v1 APIs. requirements.txt and
requirements-dev.txt are deleted rather than re-exported: pyproject.toml +
uv.lock are the one dependency source, and the docs now say `uv sync`.

minipada/ros2_pre-commit's two checks are vendored to
tools/ci/pre-commit/ros2_package_checks.py: upstream's unbounded
`find -name setup.py` walks into the ./.venv uv sync creates and checks numpy's
setup.py against the DC version, and both hooks shell out to xmllint, which the
runner image doesn't ship. Reading package.xml with ElementTree -- the parser the
check-xml hook already uses -- is what leaves this job with no system
dependencies at all.

CLAUDE.md gains a Comments rule (headers one to three lines, why not what, long
rationale in progress.txt or the PR), prompted by this branch's first draft of
format.yaml carrying a 22-line header essay.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015aZ8Ng8bK4mK9keCKxPzwX
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
@Minipada
Minipada force-pushed the chore/prek-uv-migration branch from 95ea6b1 to 78a3fbc Compare August 17, 2026 18:09
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.54%. Comparing base (cfb94dd) to head (78a3fbc).
⚠️ Report is 1 commits behind head on jazzy.

Additional details and impacted files
@@            Coverage Diff             @@
##            jazzy     #350      +/-   ##
==========================================
- Coverage   67.55%   67.54%   -0.00%     
==========================================
  Files          95       95              
  Lines        5885     5884       -1     
==========================================
- Hits         3975     3974       -1     
  Misses       1910     1910              
Flag Coverage Δ
cpp-jazzy 67.54% <100.00%> (-<0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Minipada
Minipada merged commit 7062dfa into jazzy Aug 17, 2026
6 checks passed
@Minipada
Minipada deleted the chore/prek-uv-migration branch September 2, 2026 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant