Differentiable Max and folding: Can't take the gradients of maximum$108, which depend on intermediate values. Use a scan (which saves intermediate results) instead

[soufianekhiat]

Hi,
I try to got the adjoint (gradient) of a function using max, I got this message:

Error: Can't take the gradients of maximum$108, which depend on intermediate values. Use a scan (which saves intermediate results) instead.

What is this "scan" we are talking about?

I could try to have "differentiable" max by replacing:

MyMax( x, y ) := (1/2)( x + y + abs( x - y ) )
// or even
MyMax( x, y ) := (1/2)( x + y + sqrt( ( x - y )^2 ) )

But I wasn't able to find a neat way to implement it.
Let say I have a RDom and a Func I want to have something like:

result() = ForEach( MyMax, myDom );

or

result( x ) = ForEach( MyMax, myDom );

On Mathematica that would be something like:

Fold[ MyMax, x0, { x1, x2, x3, ... } ]

Which will generate

MyMax[ x0, MyMax[ x1, MyMax[ x2, ... ] ] ]

For now I wasn't able to find a to to implement this strategy to explore different scheduling like pyramide:

MyMax( MyMax( x0, x1 ), MyMax( x2, x3 ) )

vs

MyMax( x0, MyMax( x1, MyMax( x2, x3 ) ) )