SFN parser: support AWS JSONPath filter dialect (&&/|| and empty filter results) - #10095
Open
koteshyelamati wants to merge 3 commits into
Open
SFN parser: support AWS JSONPath filter dialect (&&/|| and empty filter results)#10095koteshyelamati wants to merge 3 commits into
koteshyelamati wants to merge 3 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10095 +/- ##
=======================================
Coverage 93.28% 93.29%
=======================================
Files 1336 1336
Lines 121679 121770 +91
=======================================
+ Hits 113508 113604 +96
+ Misses 8171 8166 -5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
I would not recommend to merge this as is - patching just these two cases without extending testing corpus is future error prone. I believe swapping whole jsonpath engine is more future proof solution here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10078
The Step Functions parser backend evaluates ASL JSONPath through
jsonpath_ng.ext, which does not implement two features of AWS Step Functions' JSONPath dialect. Both appear in real-world (CDK-generated) state machines and surface asStates.Runtimeerrors:&&/||filter conjunctions. AWS SFN filter expressions use&&/||(e.g.$.items[?(@.a == true && @.b == true)]); jsonpath_ng only accepts single&and raisesJsonPathParserErroron the doubled form, and its filter grammar has no disjunction at all.[]. On AWS, a filter that matches nothing produces[];extract_jsonraisedNoSuchJsonPathError(it returned[]only for slice/wildcard paths).Rather than swapping the JSONPath engine (a larger dependency decision), this implements the dialect on top of jsonpath_ng inside
extract_json:&&is translated to jsonpath_ng's&, skipping string literals.||is handled by expanding the filter into one path per disjunct (respecting AWS's precedence:&&binds tighter than||), evaluating each, and merging matches back into document order with duplicates removed. Nothing changes for paths without||.[], consistent with the existing slice/wildcard behaviour.Operators inside string literals (e.g.
[?(@.v == 'x && y')]) are preserved by quote-aware scanning in both transformations.Added unit tests covering conjunctions, disjunctions (including precedence, chaining and document order), quoted literals, empty filter results, and the pre-existing behaviours (definite paths still raise
NoSuchJsonPathErrorwhen missing, wildcard/index access unchanged). All existing parser test suites pass.Together with #10093 and #10094 this completes the set of three issues (#10076, #10077, #10078) blocking the
states:startExecution.syncpath for CDK-generated state machines.