Skip to content

Introduce Array API - #25

Merged
mrecachinas merged 12 commits into
mrecachinas:masterfrom
RandomNameUser:array_api
Feb 16, 2026
Merged

Introduce Array API#25
mrecachinas merged 12 commits into
mrecachinas:masterfrom
RandomNameUser:array_api

Conversation

@RandomNameUser

Copy link
Copy Markdown
Contributor

Added cleaner APIs for array handling (find first, best, all matches)
Added GIL release for array ops
Added docs for array ops to Readme
Added tests for array ops
Added explicit hamming_distance_byte to have the same interface as hamming_distance_string. Functionally the find first array op does the same thing, but this being Python I'm weary of having two apparently interchangeable functions that have different return types, that just smells like trouble. ;)

It all seems to work fine, as far as I can tell. The only thing I don't understand is the benchmarking: the best and all array ops are about 2 orders of magnitude faster than the first one. Either the compiler does very high-level optimization magic, or something is wrong that I can't see...

Thanks!

RandomNameUser and others added 3 commits February 10, 2023 16:19
Added GIL release for array functions
Added check_bytes_within_dist to mirror check_hexstrings_within_dist
Fixed 4 space/8 space tab problems

@mrecachinas mrecachinas left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine. I only noted two changes before we'll want to merge. I'm not certain off-hand why you're observing that benchmark behavior, but I'd have to look further into it.

I'm comfortable merging and cutting a release after you make those changes if it suits your needs. 👍

Comment thread hexhamming/_version.h Outdated
Comment thread hexhamming/python_hexhamming.cc Outdated
@mrecachinas
mrecachinas requested a review from Copilot November 30, 2024 15:51

This comment was marked as outdated.

@mrecachinas
mrecachinas requested a review from Copilot September 6, 2025 20:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds cleaner APIs for array handling by introducing three new functions for finding matches within arrays of byte data. The changes include GIL release for array operations, comprehensive test coverage, documentation updates, and benchmarking support for the new functionality.

Key changes:

  • Added three new array operation functions: first, best, and all matches within a specified Hamming distance
  • Introduced check_bytes_within_dist function for byte-to-byte comparison with explicit naming
  • Added GIL release mechanisms for improved performance in multi-threaded environments

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
test/test_hexhamming.py Comprehensive test coverage for new array APIs with edge cases and benchmarking
hexhamming/python_hexhamming.cc Implementation of new wrapper functions with GIL release and improved error handling
hexhamming/_version.h Version bump to 2.3.0 reflecting new functionality
README.rst Documentation for array APIs with examples and usage tips

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread hexhamming/python_hexhamming.cc
Comment thread README.rst Outdated
Comment thread hexhamming/python_hexhamming.cc Outdated
Comment thread hexhamming/python_hexhamming.cc
Comment thread hexhamming/python_hexhamming.cc
Comment thread hexhamming/python_hexhamming.cc Outdated
mrecachinas and others added 7 commits September 6, 2025 19:18
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Fix missing return NULL after PyErr_NoMemory in check_bytes_arrays_all_within_dist
- Fix memory leak of out buffer on error paths
- Add missing Py_DECREF(tup) after PyList_Append
- Optimize _best to use early-termination check before computing full distance
- Optimize _all to use early-termination check before computing full distance
- Replace tabs with spaces for consistent formatting
The invalid language='c++11' was silently ignored by setuptools, causing
the extension to not link against libstdc++. This resulted in undefined
symbol errors for operator new[]/delete[] (_Znam/_ZdaPv) on Linux CI.
Resolved conflicts in test/test_hexhamming.py:
- Kept PR's new array API imports (first, best, all, check_bytes_within_dist)
- Applied master's black formatting style
- Kept PR's renamed function names (first/best/all variants)
@mrecachinas
mrecachinas self-requested a review February 16, 2026 16:47
@mrecachinas mrecachinas changed the title Array api Introduce Array API Feb 16, 2026
@mrecachinas

mrecachinas commented Feb 16, 2026

Copy link
Copy Markdown
Owner

Hey, I dug into the benchmark anomaly -- turns out it was a combination of bugs that we've now fixed:

  1. Py_BuildValue("i") was silently truncating the int64/uint64 return values to 32-bit C ints, which could produce garbage values that confused the benchmark harness
  2. The best function had an overflow bug (best_dist = max_dist + 1) that, after finding the first match, set the threshold to -1 — causing it to inadvertently skip the early-exit check and compute raw distances for every remaining element

After fixing those (plus a malloc safety fix and a NEON edge case), benchmarks now show the expected ordering: first is fastest (early exit), then all, then best (has to scan everything twice to find the minimum). Numbers look right -- first is ~5-18x faster than the others depending on the test case.

@mrecachinas
mrecachinas merged commit b02e8a4 into mrecachinas:master Feb 16, 2026
12 checks passed
mrecachinas added a commit that referenced this pull request Feb 16, 2026
* Fix security and correctness issues in array API

- Fix Py_BuildValue format specifiers: use 'L' for int64_t and 'K' for
  uint64_t instead of 'i' which silently truncates 64-bit values
- Fix signed integer overflow UB in best_within_dist when max_dist is
  large: use -1 sentinel instead of max_dist+1
- Replace new[]/delete[] with malloc/free + NULL check to prevent
  uncaught std::bad_alloc from crashing the process
- Fix NEON bug: max_dist > 0 should be max_dist >= 0 so that max_dist=0
  correctly delegates to the native path

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add Python 3.14 to CI build matrix

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants