|
| 1 | +# hexhamming - Fast Hamming Distance Calculation |
| 2 | + |
| 3 | +hexhamming is a Python C extension module that provides blazingly fast bitwise Hamming distance calculation for hexadecimal strings and byte arrays. It uses vectorized algorithms (SSE4.1, AVX2, ARM NEON) for optimal performance. |
| 4 | + |
| 5 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 6 | + |
| 7 | +## Working Effectively |
| 8 | + |
| 9 | +### Essential Setup Commands |
| 10 | +Run these commands in sequence to set up the development environment: |
| 11 | + |
| 12 | +```bash |
| 13 | +# Install development dependencies |
| 14 | +pip3 install -r requirements-dev.txt |
| 15 | +# Takes ~30 seconds - includes pytest, black, pytest-benchmark |
| 16 | + |
| 17 | +# Build and install the package (RECOMMENDED METHOD) |
| 18 | +pip3 install . |
| 19 | +# Takes 2-3 minutes. NEVER CANCEL - C++ compilation requires time. |
| 20 | +# Set timeout to 5+ minutes for build commands. |
| 21 | +``` |
| 22 | + |
| 23 | +### Alternative Build Methods |
| 24 | +If the recommended method fails due to network issues: |
| 25 | + |
| 26 | +```bash |
| 27 | +# Legacy build method (always works offline) |
| 28 | +python3 setup.py install --user |
| 29 | +# Takes 2-3 minutes. NEVER CANCEL. Shows deprecation warnings but works reliably. |
| 30 | +``` |
| 31 | + |
| 32 | +### Running Tests |
| 33 | +```bash |
| 34 | +# Run the complete test suite with benchmarks |
| 35 | +python3 -m pytest -vls . |
| 36 | +# Takes ~27 seconds. NEVER CANCEL - includes performance benchmarks. |
| 37 | +# Set timeout to 2+ minutes for test commands. |
| 38 | + |
| 39 | +# Quick test run (no benchmarks) |
| 40 | +python3 -m pytest test/ -k "not bench" |
| 41 | +# Takes ~5 seconds for functional tests only. |
| 42 | +``` |
| 43 | + |
| 44 | +### Code Quality and CI Requirements |
| 45 | +```bash |
| 46 | +# CRITICAL: Check code formatting before committing |
| 47 | +black --check . |
| 48 | +# Returns exit code 1 if formatting needed |
| 49 | + |
| 50 | +# Fix code formatting (if needed) |
| 51 | +black . |
| 52 | +# Reformats Python files to match project standards |
| 53 | + |
| 54 | +# Validate package manifest |
| 55 | +python3 -m pip install check-manifest |
| 56 | +python3 -m check_manifest |
| 57 | +# Verifies all files are properly included in package |
| 58 | +``` |
| 59 | + |
| 60 | +## Validation Scenarios |
| 61 | + |
| 62 | +### ALWAYS Test These After Making Changes |
| 63 | +After any code modifications, run these validation scenarios: |
| 64 | + |
| 65 | +```bash |
| 66 | +# 1. Basic string functionality test |
| 67 | +python3 -c " |
| 68 | +from hexhamming import hamming_distance_string |
| 69 | +result = hamming_distance_string('deadbeef', '00000000') |
| 70 | +assert result == 24, f'Expected 24, got {result}' |
| 71 | +print('✓ String hamming distance: PASS') |
| 72 | +" |
| 73 | + |
| 74 | +# 2. Bytes functionality test |
| 75 | +python3 -c " |
| 76 | +from hexhamming import hamming_distance_bytes |
| 77 | +result = hamming_distance_bytes(b'\xde\xad\xbe\xef', b'\x00\x00\x00\x00') |
| 78 | +assert result == 24, f'Expected 24, got {result}' |
| 79 | +print('✓ Bytes hamming distance: PASS') |
| 80 | +" |
| 81 | + |
| 82 | +# 3. Within distance check test |
| 83 | +python3 -c " |
| 84 | +from hexhamming import check_hexstrings_within_dist |
| 85 | +result1 = check_hexstrings_within_dist('ffff', 'fffe', 2) |
| 86 | +result2 = check_hexstrings_within_dist('ffff', '0000', 2) |
| 87 | +assert result1 == True, f'Expected True, got {result1}' |
| 88 | +assert result2 == False, f'Expected False, got {result2}' |
| 89 | +print('✓ Within distance check: PASS') |
| 90 | +" |
| 91 | + |
| 92 | +# 4. Algorithm switching test |
| 93 | +python3 -c " |
| 94 | +from hexhamming import set_algo, hamming_distance_string |
| 95 | +set_algo('classic') |
| 96 | +result = hamming_distance_string('abc', 'def') |
| 97 | +assert isinstance(result, int), f'Algorithm switch failed' |
| 98 | +print('✓ Algorithm switching: PASS') |
| 99 | +" |
| 100 | +``` |
| 101 | + |
| 102 | +### Build Validation Scenarios |
| 103 | +Test that your changes don't break the build: |
| 104 | + |
| 105 | +```bash |
| 106 | +# Clean build test (removes any cached artifacts) |
| 107 | +rm -rf build/ dist/ *.egg-info/ test/__pycache__/ .pytest_cache/ |
| 108 | +pip3 install . |
| 109 | +# Takes 2-3 minutes. NEVER CANCEL. |
| 110 | + |
| 111 | +# Verify installation worked |
| 112 | +python3 -c "import hexhamming; print('✓ Import successful')" |
| 113 | +``` |
| 114 | + |
| 115 | +## Common Build Issues and Solutions |
| 116 | + |
| 117 | +### Network Connectivity Problems |
| 118 | +If `pip install` or `python -m build` fail with network errors: |
| 119 | +- Use the legacy method: `python3 setup.py install --user` |
| 120 | +- This method works completely offline after initial dependency installation |
| 121 | + |
| 122 | +### Build Dependencies Missing |
| 123 | +If you get compiler errors: |
| 124 | +```bash |
| 125 | +# Install build essentials (Ubuntu/Debian) |
| 126 | +sudo apt-get update && sudo apt-get install build-essential python3-dev |
| 127 | + |
| 128 | +# On other systems, ensure you have a C++ compiler and Python headers |
| 129 | +``` |
| 130 | + |
| 131 | +### Test Failures |
| 132 | +If tests fail after your changes: |
| 133 | +- Focus only on failures related to your changes |
| 134 | +- Ignore unrelated benchmark timing variations |
| 135 | +- Always run the validation scenarios above to verify core functionality |
| 136 | + |
| 137 | +## Performance Expectations |
| 138 | + |
| 139 | +### CRITICAL Timing Information - NEVER CANCEL |
| 140 | +- **Development setup**: 30 seconds to 2 minutes |
| 141 | +- **Package build**: 2-3 minutes (C++ compilation takes time) |
| 142 | +- **Full test suite**: ~27 seconds (includes performance benchmarks) |
| 143 | +- **Code formatting**: 1-2 seconds |
| 144 | +- **Manifest check**: 5-10 seconds |
| 145 | + |
| 146 | +**NEVER CANCEL builds or long-running commands.** C++ compilation and performance benchmarks require time to complete. Always set timeouts of 5+ minutes for builds and 2+ minutes for tests. |
| 147 | + |
| 148 | +## Repository Structure |
| 149 | + |
| 150 | +### Key Files and Directories |
| 151 | +``` |
| 152 | +hexhamming/ |
| 153 | +├── README.rst # Main documentation and usage examples |
| 154 | +├── setup.py # Build configuration with C++ extension |
| 155 | +├── requirements-dev.txt # Development dependencies (pytest, black, etc.) |
| 156 | +├── hexhamming/ # C++ source code directory |
| 157 | +│ ├── python_hexhamming.cc # Main C++ implementation |
| 158 | +│ ├── python_hexhamming.h # Header with vectorized algorithms |
| 159 | +│ └── _version.h # Version information |
| 160 | +├── test/ |
| 161 | +│ └── test_hexhamming.py # Comprehensive test suite with benchmarks |
| 162 | +├── .github/workflows/ |
| 163 | +│ └── pythonpackage.yml # CI/CD pipeline using cibuildwheel |
| 164 | +└── MANIFEST.in # Package file inclusion rules |
| 165 | +``` |
| 166 | + |
| 167 | +### Important Code Locations |
| 168 | +- **Algorithm implementations**: `hexhamming/python_hexhamming.h` (lines 150-630) |
| 169 | +- **Python bindings**: `hexhamming/python_hexhamming.cc` |
| 170 | +- **Performance tests**: `test/test_hexhamming.py` (lines 100+) |
| 171 | +- **Build configuration**: `setup.py` (platform-specific optimizations) |
| 172 | + |
| 173 | +## Development Workflow |
| 174 | + |
| 175 | +### Making Changes |
| 176 | +1. **ALWAYS** run the setup commands first |
| 177 | +2. Make your code changes |
| 178 | +3. **IMMEDIATELY** test with validation scenarios |
| 179 | +4. Run `black --check .` and fix formatting if needed |
| 180 | +5. Run full test suite: `python3 -m pytest -vls .` |
| 181 | +6. Run `check-manifest` to verify package integrity |
| 182 | + |
| 183 | +### Before Committing |
| 184 | +```bash |
| 185 | +# Required checks that CI will run |
| 186 | +black --check . # Code formatting |
| 187 | +python3 -m check_manifest # Package manifest |
| 188 | +python3 -m pytest -vls . # Full test suite |
| 189 | +``` |
| 190 | + |
| 191 | +### CI/CD Information |
| 192 | +The project uses GitHub Actions with cibuildwheel for cross-platform wheel building: |
| 193 | +- Builds wheels for Linux (manylinux, musllinux), macOS, Windows |
| 194 | +- Tests on Python 3.6-3.10 |
| 195 | +- **Build time in CI**: 15-45 minutes depending on platform |
| 196 | +- **Network dependencies**: Requires PyPI access for dependencies |
| 197 | + |
| 198 | +If CI fails on build steps, it's often due to: |
| 199 | +1. Code formatting issues (run `black .`) |
| 200 | +2. Package manifest issues (run `check-manifest`) |
| 201 | +3. Test failures (run validation scenarios locally) |
| 202 | + |
| 203 | +## Troubleshooting |
| 204 | + |
| 205 | +### "Module not found" errors |
| 206 | +Ensure you've installed the package: `pip3 install .` |
| 207 | + |
| 208 | +### Compilation errors |
| 209 | +Check that you have build tools: `sudo apt-get install build-essential python3-dev` |
| 210 | + |
| 211 | +### Test failures |
| 212 | +Run validation scenarios individually to isolate issues. |
| 213 | + |
| 214 | +### Network timeouts |
| 215 | +Use legacy build method: `python3 setup.py install --user` |
| 216 | + |
| 217 | +### Formatting issues |
| 218 | +Run `black .` to fix all formatting issues automatically. |
0 commit comments