QWED-MCP Audit Finding: MED-05
Severity: π‘ MEDIUM
Component: logic_engine.py β parse_statement (" is " / " is not " patterns)
Audit ID: MED-05
Re-audit of v0.2.1: Finding is NEW β distinct from #16 (status conflation). This is semantic corruption producing wrong verified=True.
Description
The natural-language is patterns silently discard the right-hand operand and misbind negation. Empirically confirmed on v0.2.1:
# logic_engine.py L75-80
if " is " in stmt:
parts = stmt.split(" is ", 1)
if parts[1].startswith("not "):
return Not(get_var(parts[0])) # "A is not B" -> NOT A (B dropped!)
return get_var(parts[0]) # "A is B" -> A (B dropped!)
Live probe results:
verify_logic_statement(premises=["A is B"], conclusion="A")
# -> verified=True (premise was silently rewritten to just "A")
verify_logic_statement(premises=["A is not B"], conclusion="not A")
# -> verified=True ("A is not B" was rewritten to NOT A; "B" vanished)
The engine returns verified=True for arguments whose premises it silently rewrote. This is worse than a parse failure: the user receives an affirmative verification of a different argument than the one submitted.
Semantic expectations that are violated:
| Input |
Parsed as |
Any reasonable reading |
A is B |
A |
A β B or A β B β never bare A |
A is not B |
Β¬A |
A β Β¬B β never bare Β¬A |
Why This Violates QWED Philosophy
- Principle 6 β No Silent Degradation: The parser quietly substitutes a weaker (and different) proposition for the submitted one, then verifies the substitute. Failures must remain visible; here the failure is invisible and crowned with
verified=True.
- Principle 2 β Fail Closed: If
is/is not semantics cannot be deterministically modeled, the pattern must be rejected as UNVERIFIABLE, not approximated.
Expected Behavior
Suggested Fix Direction
if " is " in stmt:
parts = stmt.split(" is ", 1)
lhs, rhs = get_var(parts[0]), parse_statement(parts[1])
return And(Implies(lhs, rhs), Implies(rhs, lhs)) # biconditional, or
# raise UnsupportedStatementError(...) # fail closed
Whichever semantics is chosen, add regression tests proving the RHS operand participates in the Z3 model (e.g., premises=["A is B"], conclusion="B" must verify; premises=["A is not B"], conclusion="not B" behavior must match the documented semantics).
Environment
| Field |
Value |
| QWED-MCP Version |
0.2.1 |
| Component |
Logic Verification Engine |
| File |
src/qwed_mcp/engines/logic_engine.py (L75β80) |
QWED-MCP Audit Finding: MED-05
Severity: π‘ MEDIUM
Component:
logic_engine.pyβparse_statement(" is "/" is not "patterns)Audit ID: MED-05
Re-audit of v0.2.1: Finding is NEW β distinct from #16 (status conflation). This is semantic corruption producing wrong
verified=True.Description
The natural-language
ispatterns silently discard the right-hand operand and misbind negation. Empirically confirmed on v0.2.1:Live probe results:
The engine returns
verified=Truefor arguments whose premises it silently rewrote. This is worse than a parse failure: the user receives an affirmative verification of a different argument than the one submitted.Semantic expectations that are violated:
A is BAA β BorA β Bβ never bareAA is not BΒ¬AA β Β¬Bβ never bareΒ¬AWhy This Violates QWED Philosophy
verified=True.is/is notsemantics cannot be deterministically modeled, the pattern must be rejected as UNVERIFIABLE, not approximated.Expected Behavior
A is Bmust either map to an explicit deterministic semantics (e.g., biconditionalA β B, or implication with documented direction) or be rejected as unsupported.A is not Bmust never dropB.verified=Falsewithverification_mode: "UNVERIFIABLE"(aligns with the status taxonomy in [Bug]: Logic engine conflates parse errors with logical refutations β cannot distinguish DISPROVEN from UNVERIFIABLEΒ #16), never a rewritten premise.Suggested Fix Direction
Whichever semantics is chosen, add regression tests proving the RHS operand participates in the Z3 model (e.g.,
premises=["A is B"], conclusion="B"must verify;premises=["A is not B"], conclusion="not B"behavior must match the documented semantics).Environment
0.2.1src/qwed_mcp/engines/logic_engine.py(L75β80)