Skip to content

Implement updates with folding operators (reduce/foreach) on LHS.#281

Merged
01mf02 merged 4 commits intomainfrom
fold-update
Apr 22, 2025
Merged

Implement updates with folding operators (reduce/foreach) on LHS.#281
01mf02 merged 4 commits intomainfrom
fold-update

Conversation

@01mf02
Copy link
Copy Markdown
Owner

@01mf02 01mf02 commented Apr 22, 2025

This PR enables updates with folding operators (reduce and foreach) on the left-hand side of updates.
This is something that jq cannot currently do.

Motivation

The motivating case for this PR was the implementation of getpath. In jq, we can use getpath on the left-hand side of updates:

$ jq -nc '[[2, 3]] | getpath([0, 1]) += 1'
[[2,4]]

In jq, getpath is a builtin filter implemented in C.
In jaq, I tried to first implement getpath as definition, as such:

def getpath($p): if $p != [] then .[$p[0]] | getpath($p[1:]) end;

This is a more convoluted version of the following:

def getpath($path): reduce $path[] as $p (.; .[$p]);

The difference between the two versions is that the first version can be used on the left-hand side of updates, whereas the second version cannot:

$ jq -nc 'def getpath($p): if $p != [] then .[$p[0]] | getpath($p[1:]) end; [[2, 3]] | getpath([0, 1]) += 1'
[[2,4]]
$ jq -nc 'def getpath($path): reduce $path[] as $p (.; .[$p]); [[2, 3]] | getpath([0, 1]) += 1'
jq: error (at <unknown>): Invalid path expression near attempt to iterate through [0,1]

Unfortunately, while the first version can be used on the left-hand side of updates, it is significantly slower than the second version.

I then tried to implement getpath as native filter, like it is done in jq, but this got quite convoluted and required a breaking API change for achieving optimal performance (3eede6c).

This got me thinking: How about finally implementing update support for reduce/foreach? That way, we could use the more performant & simpler second definition of getpath and use it on the left-hand side of updates.

With this PR, jaq can do it:

$ jaq -nc 'def getpath($path): reduce $path[] as $p (.; .[$p]); [[2, 3]] | getpath([0, 1]) += 1'
[[2,4]]

Implementation

This is a relatively straightforward implementation of what I described already one year ago in my formal specification of the jq language (section "Update semantics" > "Folding").

Acknowledgements

Thanks to @osevill for having provided a convincing use case for getpath #277. This greatly motivated me to finally implement this functionality that was already a long time on my roadmap.

@01mf02 01mf02 merged commit 32e0cf3 into main Apr 22, 2025
3 checks passed
@01mf02 01mf02 deleted the fold-update branch April 22, 2025 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant