feat: enforce output_model validation in EventLoopNode (fixes #5929)#6005
Open
Waryjustice wants to merge 1 commit intoaden-hive:mainfrom
Open
feat: enforce output_model validation in EventLoopNode (fixes #5929)#6005Waryjustice wants to merge 1 commit intoaden-hive:mainfrom
Waryjustice wants to merge 1 commit intoaden-hive:mainfrom
Conversation
Wire NodeSpec.output_model into the set_output handling path so that
structured-output validation is actually enforced at runtime.
Previously, output_model and max_validation_retries were defined on
NodeSpec (and tested for existence) but were never read by the runtime.
A node declaring output_model=OrderSchema behaved identically to one
without it, silently passing malformed LLM outputs into shared memory.
Changes:
- Import OutputValidator in event_loop_node.py
- Track _validation_retry_count in the main execute() loop
- In the ACCEPT path (after missing-keys guard), call
OutputValidator.validate_with_pydantic(accumulator.to_dict(), output_model)
* On ValidationError: inject '[Output validation failed]: <feedback>'
into conversation history and RETRY (up to max_validation_retries)
* On exhausted retries: ESCALATE with structured error detail and
log via runtime_logger (log_step + log_node_complete)
* On success: fall through to the normal memory-write path unchanged
- Nodes without output_model are completely unaffected (zero behaviour change)
- No new dependencies — pydantic and OutputValidator already present
Extend core/tests/test_pydantic_validation.py with TestRuntimeValidationPath:
tests that mirror the exact validation logic executed in the ACCEPT path,
covering: valid pass-through, invalid feedback generation, injection message
format, retry budget tracking, zero-retries escalation, type coercion
failures, constraint violations, and default-value preservation.
Fixes aden-hive#5929
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR Closed - Requirements Not MetThis PR has been automatically closed because it doesn't meet the requirements. PR Author: @Waryjustice To fix:
Exception: To bypass this requirement, you can:
Micro-fix requirements (must meet ALL):
Why is this required? See #472 for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #5929.
NodeSpec.output_model and max_validation_retries were defined and tested, but never read by the runtime. A node declaring output_model=OrderSchema behaved identically to one without it, silently passing malformed LLM outputs into shared memory.
This PR wires output_model into the ACCEPT path of EventLoopNode.execute().
Changes
\core/framework/graph/event_loop_node.py\
untime_logger, and return a failed \NodeResult\
\core/tests/test_pydantic_validation.py\
Behaviour unchanged for existing nodes
Nodes that do not set \output_model\ take the \if ctx.node_spec.output_model is not None\ fast path and are completely unaffected.