Fix silent feature corruption: register-as-number parsing, x64 negative masks, duplicate string/stack-string features - #25
Open
mnaza wants to merge 2 commits into
Conversation
Owner
|
All 4 pull requests has conflicts |
- parse_operand_to_number: require a leading digit for h-suffixed and bare hex literals — register names ah/bh/ch/dh parsed as 0xA-0xD and hex-looking labels (beef, face) parsed as numbers. - mask negative immediates at the function's bitness (was always u32, truncating x64 values like mov rax, -1 to 0xFFFFFFFF). - emit stack string characteristic once per basic block (was pushed per instruction past the threshold with no break). - drop the duplicate plain-ASCII pass from extract_unicode_strings — extract_file_strings already runs extract_ascii_strings alongside, so every ASCII string was emitted twice. Adds 3 regression tests (16 total, all passing).
mnaza
force-pushed
the
fix/extractor-feature-corruption
branch
from
July 31, 2026 08:57
29ffe26 to
bc3b51a
Compare
Contributor
Author
|
Corrected, now all 4 mergable |
Owner
|
conflicts.. |
Owner
|
still |
…ture-corruption # Conflicts: # CHANGELOG.md
Contributor
Author
|
Now should merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24.
Four extraction bugs that silently emit wrong or duplicate features. 3 regression tests added (16 pass, was 13).
1. Registers
ah/bh/ch/dhand hex-looking labels parsed as numbersparse_operand_to_numbercase 2 stripped thehsuffix and parsed the rest as hex:"ah"→"a"→Some(10), somov al, ahemittedNumber(0xA). Case 5 (bare hex fallback) also accepted labels likebeef/face. Both paths now require a leading digit — the Intel convention for hex literals (0ABhkeeps working;ah,beefdo not parse).2. Negative immediates masked to 32 bits on x64
(s as u32) as i128—mov rax, -1emittedNumber(0xFFFFFFFF)instead ofNumber(0xFFFFFFFFFFFFFFFF). Newmask_to_bitnesshelper follows the function's bitness.3.
stack stringpushed once per remaining instructionThe push sat inside the instruction loop with no
break, so a block crossing the threshold got one duplicate characteristic per remaining instruction. Now emitted once per basic block.4. Every ASCII string emitted twice
extract_file_stringscallsextract_ascii_stringsandextract_unicode_strings— but the latter also ran a plain-ASCII pass ([\x20-\x7E]{4,}, the same printable class). Each ASCII string landed twice in the string feature set: double matching work for every string rule and duplicate VAs in feature maps. The Unicode extractor is now UTF-16-only; ASCII coverage stays withextract_ascii_strings(whose class is slightly wider — includes\t).Out of scope (noticed while testing, pre-existing)
Case 3 of
parse_operand_to_number(-0x10/+0x10sign forms) is dead code:RE_NUMBER_HEXcaptures the number with the0xprefix, andi128::from_str_radixrejects it — so signed hex immediates never parse. Left alone here to keep the PR focused; happy to file a follow-up if you want it fixed.Clippy clean,
cargo fmtapplied, CHANGELOG entry under[Unreleased]. Based on master; independent of the other open PRs (#17 touches the same file but different regions; only the[Unreleased]CHANGELOG heading is shared).