Skip to content

ci: GitHub Actions (Ubuntu, uv, ruff, pytest) + repo lint cleanup - #2

Merged
ahrzb merged 8 commits into
masterfrom
chore/ci-and-lint
Jul 20, 2026
Merged

ci: GitHub Actions (Ubuntu, uv, ruff, pytest) + repo lint cleanup#2
ahrzb merged 8 commits into
masterfrom
chore/ci-and-lint

Conversation

@ahrzb

@ahrzb ahrzb commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Sets up CI, and gets the repo lint-clean first so the lint gate is green from day one.

Three commits

  1. style: apply ruff format + autofix across the repo — mechanical, no logic changes. ruff format + ruff check --fix over the whole tree (44 lint errors + 19 unformatted files → 0), plus a few manual line wraps for over-length string literals. scripts/gen_datafusion_catalogue.py is E501-exempted (it emits long markdown verbatim). Listed in .git-blame-ignore-revs.
  2. chore: add .git-blame-ignore-revs — so git blame skips the reformat. GitHub applies it automatically; locally: git config blame.ignoreRevsFile .git-blame-ignore-revs.
  3. ci: add GitHub Actions workflow — the CI itself.

CI (.github/workflows/ci.yml)

Ubuntu, on push-to-master and PRs. Because the package is a pyo3/maturin extension, it installs a Rust toolchain, then uv sync --locked (builds the extension + installs locked deps, provisioning Python 3.14), then ruff check · ruff format --check · pytest — all hard gates. Rust + uv build caches keep it fast.

Decisions

  • Ubuntu only (per request) — the differential harness is the oracle-checked, platform-neutral gate. A Windows matrix job can be added later if the native/float behavior needs it.
  • Lint enforced (per request) — hence the cleanup commit; otherwise the gate would be red on existing debt.

Caveat

I can't run GitHub Actions locally, so the first run on GitHub is the real verification. Most-likely first-run snags: Python 3.14 wheel availability for a dep on Linux, or an action-version bump. Locally on this branch: ruff check clean, ruff format --check clean, 488 passed / 16 skipped.

🤖 Generated with Claude Code

ahrzb and others added 8 commits July 20, 2026 02:53
@ahrzb @claude
style: apply ruff format + autofix across the repo
2708018 
Mechanical, no logic changes. Runs `ruff format` and `ruff check --fix` over the
whole tree, plus a few manual line wraps for over-length string literals, so the
repo is lint-clean and CI can gate on ruff. `scripts/gen_datafusion_catalogue.py`
is E501-exempted (it emits long markdown verbatim).

This commit is listed in .git-blame-ignore-revs so `git blame` skips it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
chore: add .git-blame-ignore-revs for the bulk reformat
4076c29 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: add GitHub Actions workflow
03825e3 
Runs on Ubuntu for push-to-master and PRs: installs a Rust toolchain (the
package is a pyo3/maturin extension), uv sync --locked to build it and install
locked deps, then ruff check, ruff format --check, and pytest. Rust and uv
build caches keep it fast.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: retrigger
a3bbe9e 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: retrigger (2)
1a3b07b 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: bump checkout to v5 and setup-uv to v6 (Node 20 deprecation)
8e0740e 
GitHub is force-running Node 20 actions on Node 24 and warning about it.
Both bumped to their Node 24 majors; no behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: bump setup-uv to v7 (Node 24)
9e7272c 
v6 still targets node20, so the deprecation warning survived the earlier bump.
v7 is the newest MOVING major tag and runs on node24; v8 publishes only exact
tags (no v8 major tag), which would pin us to a patch that goes stale.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb @claude
ci: run lint and formatting through pre-commit
bc8a582 
Add .pre-commit-config.yaml (ruff-check + ruff-format) and make CI run
`pre-commit run --all-files` instead of invoking ruff directly, so the local
hooks and the CI gate are the same thing rather than two copies that drift.

The ruff rev is pinned to the version locked in uv.lock; bump both together.
pre-commit added to the dev group so CI uses the locked version too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ahrzb
ahrzb merged commit d9b5606 into master Jul 20, 2026
1 check passed
@ahrzb ahrzb mentioned this pull request Jul 20, 2026
Merged
@ahrzb
ahrzb deleted the chore/ci-and-lint branch July 23, 2026 16:33
This was referenced Jul 24, 2026
Closed
Merged
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude
feat(specializer): static-only queries become constant emitters — AC #2…
0782bf1 
… (M-lower stretch 6)

A query whose driving relation is a static table is evaluated ONCE at
build time by DuckDB itself; infer() returns the fixed rows (re-validated
through the output model per call) and nothing dynamic remains — no IR,
no probes, no filters. The fallback self-validates: it fires on
Unsupported/Parse errors and a dynamic query always references the row
table, which DuckDB does not know, so evaluation fails and the original
clean error surfaces unchanged. Bind errors stay hard. Bonus breadth:
aggregation, ORDER BY, and DuckDB-dialect-beyond-sqlparser all work on
the static-only path because DuckDB is the evaluator. Corpus: 53 match /
625 clean-unsupported / 0 FAIL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude @claude-agent-ahrzb
feat(specializer): static-only queries become constant emitters — AC #2…
f08b03e 
… (M-lower stretch 6)

A query whose driving relation is a static table is evaluated ONCE at
build time by DuckDB itself; infer() returns the fixed rows (re-validated
through the output model per call) and nothing dynamic remains — no IR,
no probes, no filters. The fallback self-validates: it fires on
Unsupported/Parse errors and a dynamic query always references the row
table, which DuckDB does not know, so evaluation fails and the original
clean error surfaces unchanged. Bind errors stay hard. Bonus breadth:
aggregation, ORDER BY, and DuckDB-dialect-beyond-sqlparser all work on
the static-only path because DuckDB is the evaluator. Corpus: 53 match /
625 clean-unsupported / 0 FAIL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude
feat(specializer): cranelift backend — full IR coverage, 500-seed dif…
0362b37 
…ferential green, wired behind DuckDBInferFn (TASK-44 stretches 1-3)

One JIT'd function per program, called per row: extern "C"
fn(*mut Cx) -> i64 (0 emit / 1 skip / 2 helper trap / 3+k term trap).
Coverage-first: float arith, int compares, logic, select, itof, fabs,
and consts map inline to CLIF; everything nontrivial — checked int
arith, all string ops, loads/stores, statics, probes — calls extern
helpers that delegate to the interpreter's own semantic functions
(casemap, substr_window, duck_fcmp, DuckF64, the arena), so the
backends cannot drift where they share code. Strings travel as
(offset, length) i64 pairs; trap_flag at Cx offset 0 is checked after
every fallible helper call. CraneliftFn owns a compiled InterpFn for
input/state checks, prepared statics, and the always-available
fallback.

The 500-seed random-IR differential (fuzz_cranelift_agrees_with_
interpreter) demands byte-identical outputs and traps — it caught two
real bugs before landing: load.opt and sload.opt must NORMALIZE
payloads to type defaults under a false flag, exactly as the
interpreter documents.

DuckDBInferFn now compiles cranelift-first with interpreter fallback
(AC #2) and exposes .backend; the whole pytest suite — 32 duck_check
differentials and the 678-case corpus replay at 0 FAILs — now
exercises the JIT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@claude-agent-ahrzb claude-agent-ahrzb Bot mentioned this pull request Jul 26, 2026
Merged
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude @claude-agent-ahrzb
feat(specializer): cranelift backend — full IR coverage, 500-seed dif…
fed084d 
…ferential green, wired behind DuckDBInferFn (TASK-44 stretches 1-3)

One JIT'd function per program, called per row: extern "C"
fn(*mut Cx) -> i64 (0 emit / 1 skip / 2 helper trap / 3+k term trap).
Coverage-first: float arith, int compares, logic, select, itof, fabs,
and consts map inline to CLIF; everything nontrivial — checked int
arith, all string ops, loads/stores, statics, probes — calls extern
helpers that delegate to the interpreter's own semantic functions
(casemap, substr_window, duck_fcmp, DuckF64, the arena), so the
backends cannot drift where they share code. Strings travel as
(offset, length) i64 pairs; trap_flag at Cx offset 0 is checked after
every fallible helper call. CraneliftFn owns a compiled InterpFn for
input/state checks, prepared statics, and the always-available
fallback.

The 500-seed random-IR differential (fuzz_cranelift_agrees_with_
interpreter) demands byte-identical outputs and traps — it caught two
real bugs before landing: load.opt and sload.opt must NORMALIZE
payloads to type defaults under a false flag, exactly as the
interpreter documents.

DuckDBInferFn now compiles cranelift-first with interpreter fallback
(AC #2) and exposes .backend; the whole pytest suite — 32 duck_check
differentials and the 678-case corpus replay at 0 FAILs — now
exercises the JIT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude
feat(specializer): slot-fill output + boundary bench with generic bas…
f158298 
…eline (TASK-45 stretch 3)

Measurement killed an assumption: pydantic v2's model_construct is
pure-Python and SLOWER than model_validate (1432ns vs 882ns per row,
pydantic 2.13). The marshaller now builds output rows the way the
design doc meant — object.__new__ + direct slot writes (__dict__,
__pydantic_fields_set__, extra, private), 491ns measured — and the
bench gained a 'generic' engine (SPECIALIZER_GENERIC_BOUNDARY) as the
AC #2 baseline. Results: marshaller beats the generic boundary
1.9-2.5x on the no-op f; the specializer beats the shipping native
engine 3.7-4.2x end-to-end at n=1024 and 1.5-2.8x at n=1; the JIT-vs-
interp compute gap is finally visible through Python.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
claude-agent-ahrzb Bot pushed a commit that referenced this pull request Jul 26, 2026
@ahrzb @claude @claude-agent-ahrzb
feat(specializer): slot-fill output + boundary bench with generic bas…
5b9045f 
…eline (TASK-45 stretch 3)

Measurement killed an assumption: pydantic v2's model_construct is
pure-Python and SLOWER than model_validate (1432ns vs 882ns per row,
pydantic 2.13). The marshaller now builds output rows the way the
design doc meant — object.__new__ + direct slot writes (__dict__,
__pydantic_fields_set__, extra, private), 491ns measured — and the
bench gained a 'generic' engine (SPECIALIZER_GENERIC_BOUNDARY) as the
AC #2 baseline. Results: marshaller beats the generic boundary
1.9-2.5x on the no-op f; the specializer beats the shipping native
engine 3.7-4.2x end-to-end at n=1024 and 1.5-2.8x at n=1; the JIT-vs-
interp compute gap is finally visible through Python.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ahrzb