Add comprehensive test coverage for attack vectors and CVE patterns#221
Open
Add comprehensive test coverage for attack vectors and CVE patterns#221
Conversation
Adopt test patterns from picklescan for improved detection coverage: - test_attack_vectors.py: 150+ tests covering malicious pickle patterns across all protocols (0-5) including OS, subprocess, builtins, network, code execution, and import manipulation attacks - test_cve_patterns.py: Tests for CVE-2025-10157 (submodule bypass), CVE-2025-10156 (ZIP CRC bypass), CVE-2025-10155 (file extension bypass) - test_archive_scanning.py: Tests for malicious pickles in ZIP/TAR archives with protocol version matrix - test_benign_edge_cases.py: 165 tests for safe patterns to prevent false positives, including pure data structures, stdlib objects, custom classes, and NumPy arrays. Documents known limitations where builtins module imports (range, slice) are conservatively flagged. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Extract shared helpers into test/_helpers.py to remove duplication - Tighten ~25 severity assertions from > LIKELY_SAFE to >= LIKELY_OVERTLY_MALICIOUS - Fix test_corrupted_zip_still_scanned: remove outer try/except that swallowed failures - Fix test_submodule_bypass_os_path: use raw opcodes to preserve dotted module path - Fix 3 tests broken after rebase (range, slice, set) with protocol-aware thresholds - Parametrize 25 attack vector tests into ATTACK_VECTORS data table - Parametrize 5 extension detection tests into EXTENSION_CASES data table - Add EmptyTuple + Reduce opcodes to test_types_functiontype - Use pytest.importorskip for numpy/sklearn, tmp_path for temp files - Remove test_empty_zip_with_no_pickles (tested zipfile, not fickling) - Narrow pytest.raises in test_zip_with_non_pickle_binary to ValueError Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dguido
force-pushed
the
add-picklescan-test-patterns
branch
from
3937128 to
c53dd58
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New Test Files
test_attack_vectors.py(150+ tests)Tests malicious pickle patterns across all protocols (0-5):
os.system,os.popen,os.execv)subprocess.call,subprocess.Popen,subprocess.run)eval,exec,compile,__import__,getattr)socket.create_connection,socket.socket)runpy.run_path,cProfile.run,code.InteractiveInterpreter)importlib.import_module)marshal.loads,types.FunctionType)ctypes.CDLL,pty.spawn,pydoc.locate)test_cve_patterns.py(20+ tests)Tests for three CVEs found in picklescan:
asyncio.unix_events).bin,.pt,.pth,.txt)test_archive_scanning.py(30+ tests)Tests malicious pickles in archive formats:
test_benign_edge_cases.py(165 tests)Tests benign patterns to prevent false positives:
LIKELY_SAFE)builtinsmodule imports (range, slice) flagged as malicious (conservative approach)Test plan
🤖 Generated with Claude Code