Add measured Python batch APIs - #56
Merged
Merged
Conversation
Amortize Python FFI overhead with pairwise, multi-query, packed, and writable-buffer APIs while preserving existing semantics and guarding unsafe buffer access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Use safe chunks_exact iteration in pairwise batches. Three-run medians stayed within 1.4% of the raw-pointer baseline across measured workloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Replace raw unaligned writes into Rust-owned Python bytes with safe chunk copies. Three-run medians were unchanged or up to 1.7% faster. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Let the exact-size chunks iterator collect pairwise distances directly, removing manual pointer initialization and set_len. Three-run medians stayed within 0.8% or improved up to 3.8%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Serialize distances through exact-sized output chunks instead of raw unaligned pointers. Three-run medians stayed within 0.4% of baseline across all measured sizes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Use one pinned Py_buffer guard for readonly and writable exports, removing duplicated FFI unsafe code. Three-run into benchmarks stayed within 0.9% of the prior implementation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Keep unsafe pointer dereferencing at the actual slice construction sites while exposing the inert raw pointer through a safe accessor. Three-run into benchmarks stayed within 0.7%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Keep each pair of buffer-to-slice conversions inside one documented unsafe boundary without changing generated work. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
📊 Benchmark Comparison Results✅ All benchmarks within noise104 benchmarks compared, no significant changes detected. Legend: ✅ Faster (>5%) · |
Run both revisions sequentially on the same GitHub-hosted machine so runtime SIMD dispatch and host variability cannot create false regression reports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
Keep the GIL while writing caller-owned buffers and reject writable _into APIs on free-threaded Python, where buffer exports do not guarantee exclusive access. Packed APIs remain available for detached computation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 01071e57-12fd-4f42-bc67-e2b274d0b007
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds additive Python batch APIs that amortize PyO3/FFI overhead across pairwise distance calculations and multi-query catalog scans. Results can be returned as Python lists, packed byte strings, or written into caller-provided buffers.
The batch core is implemented in safe Rust and reuses the existing SIMD and Rayon scan machinery. Prepared catalogs were evaluated but intentionally omitted because their measured advantage did not meet the 10% acceptance gate.
Advances the Python/FFI portion of #51.
Performance
Three-run medians on an Apple M4 Max, comparing the existing Python loop with the new batch transports:
_intoAdditional scan results:
first_many: 11.03 us to 752 ns (14.7x)allpacked: 30.39 us to 3.14 us (9.7x)all_into: 30.39 us to 2.06 us (14.8x)best_many: 77.50 us to 66.57 us (1.16x)The writable-output soundness fix was measured against the previous detached implementation on the same machine. Pairwise
_intoranged from -1.3% to +2.7%, and denseall_intowas 3.3% faster, all within noise.The earlier CI report showing 3-8x regressions was caused by benchmarking base and PR on separate GitHub-hosted machines with different x86 SIMD feature sets. CI now benchmarks both revisions sequentially on the same runner; the corrected comparison reports all 104 existing benchmarks within noise.
Safety
The implementation reduced the initial batch change from 20 added
unsafeoccurrences to a net increase of two;src/batch.rscontains no unsafe code. Buffer sizes, formats, contiguity, arithmetic overflow, and input/output overlap are validated.Python writable buffer exports stabilize storage but do not provide exclusive mutation access. Writable
_intoAPIs therefore keep the GIL for the complete write on standard CPython. On free-threaded Python they raiseValueErrorand direct callers to the corresponding packed API, which retains detached computation without borrowing caller-owned writable memory.Testing
cargo test --no-default-features- 92 unit and 2 doc testspython -m pytest -q test/test_batch.py- 40 passed, 1 free-threaded-only test skippedruff check test/test_batch.pyruff format --check test/test_batch.pycargo fmt --check