Skip to content

Commit f41b9b5

Browse files
test(wl_audit): close item E mutmut survivors 55+88 + re-classify 17 as equivalent
Item G1 of v1.1 test-coverage push. Closes 2 of the 3 higher-value wl_audit.py survivors from item E (2026-05-19) with surgical mock-based unit tests in TestAuditMutationCoverageGaps: - test_authorization_header_exact_splunk_prefix kills mutant 55 (line 156: `"Splunk %s"` → `"XXSplunk %sXX"`). The existing test_post_audit_event_sets_headers uses substring `in`, which the mutated header `"XXSplunk <key>XX"` slips past. New test asserts exact equality on the full header value. - test_generic_exception_returns_non_empty_error_message kills mutant 88 (line 189: `error_msg = str(e)` → `error_msg = None` in the generic exception handler). No prior test exercised this branch (existing tests only triggered HTTPError, URLError, socket.timeout). New test injects RuntimeError via mock_urlopen.side_effect and asserts error_msg is non-None + non-empty + contains the original exception's text. Mutant 17 (line 91: kwarg-filter string `"app_context"` → `"XXapp_contextXX"`) re-classified as EQUIVALENT. The original item-E triage claimed it was killable via len(event) assertions, but closer analysis showed the loop's second write to event["app_context"] is idempotent — both line-85 and the loop's line-92 pull from the same kwargs dict, so the dict overwrite has no observable effect. Documented in MUTATION_TESTING.md. Manual verification (apply mutation → run new test → confirm assertion failure → restore): - Mutant 55 applied: test fails with `expected "Splunk MY_SESSION_KEY_12345", got "XXSplunk MY_SESSION_KEY_12345XX"` - Mutant 88 applied: test fails with `assert None is not None` - After restore: 614/614 unit tests pass (+2 from prior 612) Coverage delta: bin/wl_audit.py 84% → 89% (+5pp). Effective wl_audit kill rate: 55/90 → 57/90 (61% → 63%). Remaining 33 survivors are in lower-value classes (log/error message text, equivalent HTTP-status edge cases, integration-only paths) per the item-E triage table.
1 parent b7026dc commit f41b9b5

2 files changed

Lines changed: 127 additions & 20 deletions

File tree

docs/MUTATION_TESTING.md

Lines changed: 40 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -315,32 +315,52 @@ into 4 classes:
315315
| HTTP boundary edge cases | 68 (`200 <= status_code < 300``<= 300`) | Killable but unusual | Skip — HTTP 300 (Multiple Choices) is rarely seen and not load-bearing |
316316
| Event-building + HTTP path | 17 (kwarg filter), 55 (auth header template), 88 (None error_msg) | Killable, higher value | Defer to v1.1 item G |
317317

318-
### Higher-value survivors (deferred to item G)
318+
### Higher-value survivors (status updated 2026-05-19 during item G)
319319

320320
- **Mutant 17** (line 91): kwarg filter `("app_context", "comment")`
321-
`("XXapp_contextXX", "comment")`. Mutated version doesn't filter
322-
`app_context`, so it would be added to the event dict redundantly
323-
with the explicit `event["app_context"] = ...` write earlier. The
324-
resulting event would have `app_context` twice. Killable by
325-
asserting `len(event) == expected_field_count` in test_audit.
321+
`("XXapp_contextXX", "comment")`. **RECLASSIFIED AS EQUIVALENT.**
322+
The original triage claimed this was killable via
323+
`len(event) == expected_field_count` because the unfiltered loop
324+
would write `app_context` to the event again. Closer analysis
325+
during item G showed the second write is idempotent:
326+
`event["app_context"] = kwargs["app_context"]` overwrites the
327+
line-85 assignment with the *same* value (both pull from the same
328+
`kwargs` dict), and a dict overwrite does not add a key. No
329+
observable behavior changes for any test input. The mutant is
330+
structurally indistinguishable from the original.
326331

327332
- **Mutant 55** (line 156): HTTP Authorization header template
328-
`"Splunk %s"``"XXSplunk %sXX"`. Production code would send
329-
malformed auth header; Splunk REST API would reject with 401.
330-
Tests mock urlopen and don't inspect the actual header value.
331-
Killable by mock-asserting the header value.
333+
`"Splunk %s"``"XXSplunk %sXX"`. **CLOSED in test_audit.py
334+
`test_authorization_header_exact_splunk_prefix` (item G,
335+
2026-05-19).** The existing
336+
`test_post_audit_event_sets_headers` used `"Splunk <key>" in
337+
<header>` (substring), which the mutated `"XXSplunk <key>XX"`
338+
passes. The new test asserts exact equality on the full header
339+
value. Manually verified: applying the mutation locally produces
340+
the assertion failure
341+
`expected "Splunk MY_SESSION_KEY_12345", got "XXSplunk
342+
MY_SESSION_KEY_12345XX"`.
332343

333344
- **Mutant 88** (line 189): `error_msg = str(e)``error_msg = None`
334-
in the generic exception handler. Caller receives `(False, None)`
335-
instead of `(False, "real error text")`, masking diagnostics. Test
336-
could assert `error_msg is not None and len(error_msg) > 0` in the
337-
exception path.
338-
339-
These ~3-5 higher-value survivors are in the integration-tested HTTP
340-
send path. Closing them via unit tests would require mocking the
341-
urllib.request layer — a different style of test than the existing
342-
event-construction unit tests. Defer to v1.1 item G as part of the
343-
broader test-coverage push.
345+
in the generic exception handler. **CLOSED in test_audit.py
346+
`test_generic_exception_returns_non_empty_error_message` (item
347+
G, 2026-05-19).** No prior test exercised the generic
348+
`except Exception` branch (existing tests only triggered HTTPError,
349+
URLError, socket.timeout). The new test injects a `RuntimeError`
350+
via `mock_urlopen.side_effect` and asserts `error_msg is not None`
351+
+ non-empty + contains the original exception's text.
352+
Manually verified: applying the mutation produces `assert None
353+
is not None` failure.
354+
355+
Closing summary (item G first batch, 2026-05-19):
356+
- Mutant 17 → equivalent (triage corrected — no test possible without
357+
destroying program semantics).
358+
- Mutant 55 → killed by exact-string header assertion.
359+
- Mutant 88 → killed by mocked-`side_effect` generic-exception test.
360+
- Effective wl_audit kill rate after this batch: 57/90 → **63%**
361+
(was 55/90 = 61%). 33 survivors remain, all in the lower-value
362+
classes (log/error message text, equivalent HTTP-status edge
363+
cases, integration-only paths).
344364

345365
---
346366

tests/unit/test_audit.py

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -409,3 +409,90 @@ def test_truncation_count_message_reports_exact_dropped_count(
409409
"expected the truncation marker to report the exact "
410410
"number of dropped entries, got: {!r}".format(marker)
411411
)
412+
413+
@patch('wl_audit.urllib.request.urlopen')
414+
def test_authorization_header_exact_splunk_prefix(self, mock_urlopen):
415+
"""Kill mutmut #55 (item E, 2026-05-19): Authorization header
416+
template ``"Splunk %s"`` mutated to ``"XXSplunk %sXX"``.
417+
418+
Existing ``test_post_audit_event_sets_headers`` uses a substring
419+
``in`` assertion (``"Splunk test-session-key" in <header>``) which
420+
the mutation slips past — the malformed header
421+
``"XXSplunk test-session-keyXX"`` still contains the substring.
422+
In production the malformed header would cause Splunk's REST API
423+
to reject the request with 401 (unauthorized) and audit events
424+
would silently fail to land. Pin the header value with exact
425+
equality so any character mutation is killed.
426+
"""
427+
from wl_audit import build_audit_event, post_audit_event
428+
429+
mock_response = MagicMock()
430+
mock_response.status = 200
431+
mock_urlopen.return_value = mock_response
432+
433+
event = build_audit_event(
434+
action="added",
435+
analyst="jsmith",
436+
detection_rule="Rule 1",
437+
csv_file="file.csv",
438+
)
439+
post_audit_event("MY_SESSION_KEY_12345", event)
440+
441+
req = mock_urlopen.call_args[0][0]
442+
auth_header = req.headers["Authorization"]
443+
444+
# Exact format pin: "Splunk <session_key>" — any character
445+
# mutation in the literal prefix or its template structure
446+
# produces a different string and fails this assertion.
447+
assert auth_header == "Splunk MY_SESSION_KEY_12345", (
448+
"Authorization header must be exactly "
449+
"'Splunk MY_SESSION_KEY_12345' (no extra characters, "
450+
"no template drift). Got: {!r}".format(auth_header)
451+
)
452+
453+
@patch('wl_audit.urllib.request.urlopen')
454+
def test_generic_exception_returns_non_empty_error_message(self, mock_urlopen):
455+
"""Kill mutmut #88 (item E, 2026-05-19): generic ``except Exception``
456+
branch assigns ``error_msg = str(e)``, mutated to ``error_msg = None``.
457+
458+
The HTTPError / URLError / socket.timeout branches are covered by
459+
other tests, but the catch-all ``except Exception`` branch has no
460+
coverage. Mutating ``error_msg = str(e)`` to ``error_msg = None``
461+
makes the caller receive ``(False, None)`` instead of
462+
``(False, "real error text")``, masking diagnostic information.
463+
464+
Trigger by raising a non-{HTTPError, URLError, timeout} exception
465+
(RuntimeError) from urlopen, then assert that the returned
466+
``error_msg`` is a non-empty string containing the original
467+
exception's text.
468+
"""
469+
from wl_audit import build_audit_event, post_audit_event
470+
471+
# RuntimeError doesn't match any of the specific except clauses
472+
# (HTTPError, URLError, socket.timeout) — falls through to the
473+
# generic ``except Exception``.
474+
mock_urlopen.side_effect = RuntimeError("kaboom: unexpected runtime failure")
475+
476+
event = build_audit_event(
477+
action="added",
478+
analyst="jsmith",
479+
detection_rule="Rule 1",
480+
csv_file="file.csv",
481+
)
482+
success, error_msg = post_audit_event("SESSION_KEY", event)
483+
484+
assert success is False
485+
# Mutant 88 sets error_msg = None — explicitly assert it isn't None.
486+
assert error_msg is not None, (
487+
"error_msg must not be None — mutant 88 silently drops "
488+
"the diagnostic text from generic-exception failures."
489+
)
490+
assert isinstance(error_msg, str), (
491+
"error_msg must be a string for caller-side logging compat. "
492+
"Got type: {}".format(type(error_msg))
493+
)
494+
assert len(error_msg) > 0, "error_msg must contain diagnostic text"
495+
assert "kaboom" in error_msg, (
496+
"error_msg should preserve the original exception's text "
497+
"(str(e) behavior). Got: {!r}".format(error_msg)
498+
)

0 commit comments

Comments
 (0)