L1 and L2 norms (Euclidean and taxicab lengths)

[bob-carpenter]

Description

We should have L1 and L2 norm functions in the Stan language. They have to be implemented in the math library first, which is what this PR is for. We want the functions, but ideally we want custom derivatives.

real norm1(vector x);
real norm1(row_vector x);

real norm2(vector x);
real norm2(row_vector x);

The definitions are:

norm1(x)  =def=  sum(abs(x))
norm2(x)  =def=  sqrt(sum(square(x)))

The derivatives are:

d/dx norm1(x) = signum(x)
d/dx norm2(x) = x / norm2(x)

• implement norm1 and norm2 in the Stan Math library with autodiff test framework tests
• replace other uses that calculate vector lengths with norm2
  • unit_vector constrain
  • ???

Current Version:

v4.1.0

[tvrsimhan]

I would like to work on this issue!

[bob-carpenter]

Hi, @tvrsimhan. That'd be great. We accept pull requests for issues. Please let us know if you have any questions about how to implement it or test it (questions on the issue are fine).

There are two possible ways to do this: either by adding the functions with inefficient autodiff implementations or by adding custom gradients. We'd accept the former, but the advantage in compute speed is from the latter.

[tvrsimhan]

Thank you, @bob-carpenter! In which directory should this be implemented? I read through the guide but I couldn't locate those directories. Also, please point me to any other links I can refer to! I'll try my best and get back to you with any questions I have!

I'll try to add custom gradients, but I'll get back to you if I have any trouble with it!

[bob-carpenter]

It should follow the other functions. That means a primitive (double) implementation in stan/math/prim/fun and then one with custom gradients in stan/math/rev/fun. The primitive version needs thorough testing, then the reverse mode should be tested in the mix directory for autodiff like the other matrix functions.

You can see how dot_self is implemented as it has the same signature:

• prim: https://github.com/stan-dev/math/blob/develop/stan/math/prim/fun/dot_self.hpp
• reverse: https://github.com/stan-dev/math/blob/develop/stan/math/rev/fun/dot_self.hpp
• prim test: https://github.com/stan-dev/math/blob/develop/test/unit/math/prim/fun/dot_self_test.cpp
• autodiff test: https://github.com/stan-dev/math/blob/develop/test/unit/math/mix/fun/dot_self_test.cpp

[tvrsimhan]

Thank you for your guidance! I'll do my best!

[yizhang-yiz]

@tvrsimhan any update on this issue? Thanks.

[SteveBronder]

@tvrsimhan the stan math docs have some examples to get started with as well

https://mc-stan.org/math/getting_started.html

[rok-cesnovar]

Ah sorry, was too quick to close. #2636 addressed the first point of the issue, the second one is still open. Reopening.