Implement more matrix functions

[jiahao]

Higham and Deadman have published a list of software implementing algorithms for matrix functions, showing that Julia's library for these functions could be improved greatly relative to the state of the art.

The purpose of this issue is to discuss and track the implementation of matrix functions.

From Higham and Deadman's list:

General function evaluations

• General matrix function with derivatives of the underlying scalar function available: Schur–Parlett algorithm (Davies and Higham, 2003)
• Function of a symmetric or Hermitian matrix by diagonalization
• Condition number estimate for general functions: cond(f,A)
• matrix function - vector product evaluation: f(A) * b

Specific functions

• expm: scaling and squaring algorithm (Al-Mohy and Higham, 2009)
• logm: inverse scaling and squaring algorithm (Al-Mohy, Higham, and Relton, 2012,
  2013)
• sqrtm:
  • Schur algorithm (Björck and Hammarling, 1983)
  • real version for real matrices (Higham, 1987)
  • blocking algorithm for performance improvements (Deadman, Higham, and Ralha, 2013)
• A^t for real matrix powers: Schur– Padé algorithm (Higham and Lin, 2013) (Fixed the algorithm for powers of a matrix. #21184)
• Matrix unwinding function

Fréchet derivatives

• Exponential: scaling and squaring algorithm (Al-Mohy and Higham, 2009)
• Logarithm: inverse scaling and squaring algorithm (Al-Mohy, Higham, and Relton, 2013)
• General matrix function: complex step algorithm (Al-Mohy and Higham, 2010) or using block 2 × 2 matrix formula.
• A^t for real matrix powers: Schur– Padé algorithm (Higham and Lin, 2013)

Miscellaneous functions

• inv and det for Tridiagonal and SymTridiagonal matrices (Usmani, 1994) (Provide inv and det of Tridiagonal and SymTridiagonal matrices #5358)

[StefanKarpinski]

It would be amazing to include all of these algorithms. Thanks for putting this issue together, @jiahao.

[jiahao]

It's not clear to me what interface should be implemented for the very generic things like "function of a symmetric or Hermitian matrix".

We could define methods for elementary functions until the cows come home, but that is both tedious and namespace polluting (we'd need sinm, cosm, atanhm, etc. to disambiguate from the ordinary forms which broadcast over the elements) , and furthermore wouldn't scale well for linear combinations of functions.

Maybe we want something like applym, so that a call like applym(A->exp(A)+sin(2A)-cosh(4A), A) would evaluate the matrix expm(A)+sinm(2A)-coshm(4A)?

[StefanKarpinski]

This is one of the reasons why having all these functions be vectorized is kind of shitty. I suspect that @JeffBezanson will vehemently agree. There could be a MatrixFunctions module that you have to import from to get the matrix variants with all the same names.

[jiahao]

Sure, but that won't help with the fact that a complicated matrix function like A->expm(A)+sinm(2A)-coshm(4A) can be computed more quickly all at once than computing (expm(A))+(sinm(2A))-(coshm(4A)) by working out the individual elementary functions separately, since the underlying Schur-Parlett algorithm is essentially the same.

[StefanKarpinski]

Yes, that's a very good point.

[stevengj]

Trefethen's contour-integration method is also neat for mostly analytic functions (with known singularities), though I don't know how it compares to the state of the art (in performance and generality).

[jiahao]

@stevengj it looks like that paper is Ref. 32 of the Higham and Deadman report, and isn't currently available in any major package (although it has Matlab code).

[simonbyrne]

This seems like a good candidate for a GSOC project.

As far as an interface goes, I would suggest a macro

B = @matfun exp(A)+sin(2A)-cosh(4A)

A naive approach could simply replace the functions (e.g. exp -> expm), a more advanced approach could try to identify the common decompositions.

[GunnarFarneback]

The item "[sqrtm] real version for real matrices (Higham, 1987)" is checked. I can't see that algorithm in base. I made an attempt on it about a year ago but concluded that the complexity just wasn't worth it. I've put up the code as a gist if someone wants to have a look.

[jiahao]

Thanks for catching that. I must have ticked it by accident when writing up this issue.

[dlfivefifty]

Just tried to call logm, but wasn't implemented. Had to go to Mathematica