fix(ai_guard): fix blocked requests not appearing in APM and LLM Observability for langchain 1.x#18555
fix(ai_guard): fix blocked requests not appearing in APM and LLM Observability for langchain 1.x#18555avara1986 wants to merge 3 commits into
Conversation
Codeowners resolved as |
|
…errors The langgraph tracing handlers were changed to `except BaseException` so AI Guard's DDBlockException (a BaseException subclass) would still finish the span. But in the stream generators that catch also swallowed GeneratorExit (break/close/aclose) and CancelledError, marking normal stream teardown as span errors. Narrow the catch to (DDBlockException, Exception), mirroring the LangChain integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
avara1986
requested review from
Yun-Kim,
christophe-papazian,
emmettbutler and
rithikanarayan
| assert mock_execute_request.call_count == 3 | ||
|
|
||
|
|
||
| @requires_create_agent |
There was a problem hiding this comment.
Where is this defined?
There was a problem hiding this comment.
It's defined at the top of this same file (line 35):
requires_create_agent = pytest.mark.skipif(
LANGCHAIN_VERSION < (1, 0, 0),
reason="create_agent API introduced in langchain 1.0",
)It mirrors requires_legacy_agents right above it — langchain 1.0 dropped the legacy AgentExecutor API in favor of the langgraph-based create_agent, so the suite is version-split.
| inputs = [inputs] | ||
| final_output = func(*args, **kwargs) | ||
| except Exception: | ||
| except (DDBlockException, Exception): |
There was a problem hiding this comment.
DDBlockException is a subclass of BaseException I am assuming? What was the reason for that out of curiosity?
There was a problem hiding this comment.
Yes! AIGuardAbortError -> DDBlockException -> BaseException (not Exception). That's intentional: AI Guard's DENY/ABORT verdict needs to propagate out of user agent code without being accidentally swallowed by a generic except Exception: (which is extremely common in LLM/agent loops). Since it sits as a sibling of Exception rather than a subclass, we have to list both in the tuple here to run set_exc_info for the blocked case while still catching ordinary errors.
| # AIDEV-NOTE: catch ``DDBlockException`` explicitly (parent of ``AIGuardAbortError``) since it inherits | ||
| # from ``BaseException`` — otherwise an AI Guard abort would slip past ``except Exception:`` and the span | ||
| # would never get ``set_exc_info`` / ``finish``. We must NOT catch bare ``BaseException`` here: this wraps | ||
| # a ``yield``, so normal stream teardown (``break`` / ``close()`` / ``aclose()`` -> ``GeneratorExit``) and | ||
| # async cancellation (``CancelledError``) would otherwise be mis-reported as span errors. |
There was a problem hiding this comment.
I think this is unnecessary and explained from the code itself
3 participants
Description
AI Guard DENY/ABORT responses raise
AIGuardAbortError, which is aBaseExceptionsubclass (notException). The LangGraph pregel stream generators usedexcept Exceptionin their inner loops, sospan.finish()was never called when a block propagated — causing the entire trace to be silently dropped from APM, LLM Observability, and the AI Guard UI.Fix:
except Exception→except BaseExceptionin the 5 LangGraph generator catch sites (traced_runnable_seq_invoke,traced_runnable_seq_ainvoke,_astreamintraced_runnable_seq_astream,_streamintraced_pregel_stream,_astreamintraced_pregel_astream)DDBlockExceptionto the except tuple in the LangChain LCEL wrappers (traced_lcel_runnable_sequence,traced_lcel_runnable_sequence_async) soset_exc_infois called for blocked requestsJira: APPSEC-68351