Repository navigation
Fix parser segfault on malformed Properties + plug memory leaks - #30
Merged
Merged
Conversation
…leaks Independently confirmed three real defects (and debunked two overstated ones) raised by the third-party Rust port `extxyz/extxyz` (`extxyz-ng`), whose README criticises this C library but filed no issues. Fixes: - Segfault on malformed `Properties`: an info line whose `Properties` value does not match `name:type:count` (e.g. `Properties=species`) slips past the grammar as a plain string, after which the strtok-based parser dereferenced a NULL token (`strlen(NULL)`). Guard every `strtok(NULL, ":")` and error cleanly. - Grammar memory leak: the compiled cleri grammar was never freed (240 leaks / 13.5 KB under macOS `leaks`; matches the valgrind trace in the Rust port's README). Expose `cleri_grammar_free` to the Python binding and free the cached grammar via `atexit`; free it (and the dicts) in the C test driver. - Error-path dict leaks: mid-parse failures freed `line`/`re_str`/`props` but leaked the partially built `*info`/`*arrays`. Add `free_partial_dicts()` on every error path and `calloc` the string column so partial frees are safe. - C test driver (`test_C_main.c`): pass `comment = NULL` and a proper `error_message` buffer (were single `char`s), so it parses the info line into separate keys instead of reading the whole 2nd line as one `comment`, and add full teardown. Fixes #20. Tests: `tests/test_robustness.py` reproduces each crash/leak in a subprocess and locks in that space-padded data rows and leading-whitespace info lines (two non-bugs the Rust port alleged) parse correctly on both backends. `tests/test_bench_vs_rust.py` is a skip-by-default head-to-head benchmark. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ence gcc warning The subprocess-based robustness tests forced the repo `python/` dir onto sys.path, but that source tree lacks the build-generated `extxyz._version`, so they failed under CI (where extxyz is installed as a wheel) with `ModuleNotFoundError: No module named 'extxyz._version'`. Let the child inherit the parent's environment instead, so it imports the same extxyz the suite does (installed in CI, PYTHONPATH source tree locally). Also initialise `cur_array` at its declaration to silence a GCC -Wmaybe-uninitialized false positive introduced by the new NULL guards. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Windows extension only exports symbols listed in _extxyz.def, so `cleri_grammar_free` (a libcleri symbol) was missing there, breaking import of cextxyz and thus all test collection. Add it to the .def, and make the Python binding degrade gracefully (`_have_grammar_free`) if a build ever lacks it. Also build the test's CRASH_SIGNALS set from only the signals that exist on the platform -- `signal.SIGBUS` is absent on Windows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The third-party Rust port
extxyz/extxyz(crateextxyz-ng) markets itself against this library, citing a memory leak (with a valgrind trace) and segfaults on "misaligned input" — but filed no issues. I independently verified each claim. Three are real and fixed here; two are overstated/false and now have regression tests.Fixes
1. Segfault on malformed
Properties(NULL deref)An info line whose
Propertiesvalue doesn't matchname:type:count(e.g.Properties=species) fails the grammar's strict rule, falls through to the generic key=value rule, and is stored as a plain string. Thestrtok-based Properties parser then dereferenced a NULL token (strtok(NULL, ":")→strlen(NULL)) → SIGSEGV. Now everystrtokresult is NULL-checked and reported as a cleanExtXYZError.2. Grammar memory leak
The compiled
clerigrammar was never freed — 240 leaks / 13.5 KB under macOSleaks, matching the valgrind trace in the Rust port's README. Nowcleri_grammar_freeis exposed to the Python binding and the cached grammar is freed viaatexit; the C test driver frees it (and the dicts) too.3. Error-path dict leaks
Mid-parse failures freed
line/re_str/propsbut leaked the partially built*info/*arrays(17 leaks / 720 B on a truncated file). Addedfree_partial_dicts()on every error path; string columns are nowcalloc'd so freeing a partially filled array is safe.4. C test driver — fixes #20
test_C_main.cpassed&comment(a singlechar, non-NULL) and a single-charerror_message, so the reader treated the entire 2nd line as onecommentkey and risked a buffer overflow on errors. Now it passescomment = NULLand a realerror_messagebuffer, and tears everything down. Verified against the example in #20 —Lattice,Properties,pbc, etc. now parse as separate keys.Not bugs (regression-tested)
Tests
tests/test_robustness.py— reproduces each crash/leak in a subprocess (so a SIGSEGV can't take pytest down) and covers the two non-bugs above. 18 cases, both backends.tests/test_bench_vs_rust.py— skip-by-default head-to-head benchmark vsextxyz-ng(runs it in a separate venv via subprocess, since both packages import asextxyz).Verification
meson compile— all targets, 0 warnings.leaks— 0 leaks on valid, truncated, malformed-Properties, and multi-property files (was 240 + 17 leaks).pytest tests/— 20 passed, 2 skipped (bench).Benchmark (measured, current code with PCRE2 JIT)
extxyz-ng4.5 ms → 1.6× faster for them (not the README's "~4×", which was vs the pre-JIT legacy C).extxyz-ngfails to read it — its multi-frame reader desyncs after a few non-trivial frames. Ours reads all 200 frames in ~48 ms.Closes #20.
🤖 Generated with Claude Code