Decouple blas-src from LAPACK providers

[jedbrown]

I'd like to support a dependency tree in which some packages depend on blas-src with feature blis and others need lapack-src. While some (like openblas-src) bundle both the optimized BLAS and nearly-reference LAPACK, other BLAS implementations do not bundle LAPACK.

I'm opening this issue to get feedback on how this should be done. I think one option is to have a new x-lapack-src that depends on blas-src (so it works with any BLAS) and provides a netlib-equivalent LAPACK. My inclination is actually to do this using an f2c'd version of netlib LAPACK that we maintain for PETSc because it removes a need for a Fortran compiler to be available, but the same could readily be done for the Fortran sources. If doing this, what should the repository and feature for this be called? Other suggestions welcome.

[IvanUkhov]

I am not sure I understand the decoupling you are talking about. There are those two packages for selecting sources of BLAS and LAPACK, and one can always opt out to depend on any specific directly. Would you be able to elaborate on the problem you are facing?

[jedbrown]

So the docs (which have become inconsistent in the wiki) have been recommending that people use features to identify a provider.

[dependencies]
blas-src = { version = "0.9", features = ["blis"] }

But BLIS does not provide LAPACK so we currently can't use it with lapack-src. It needs to be combined with a LAPACK (which other implementations are bundling).

[dependencies]
lapack-src = { version = "0.9", features = ["what here?"] }

I could make a new provider, say f2c-netlib-lapack-src that provides only LAPACK (no BLAS) and then users would have

lapack-src = { version = "0.9", features = ["f2c-netlib"] }
blas-src = { version = "0.9", features = ["blis"] }

This requires only a C compiler, will build fast, and run fast. I think it will have confusing link errors (undefined or multiply defined symbols) if someone forgets blas-src or uses a blas-src that does bundles LAPACK. We're already abusing features for mutually exclusive things and I fear this will make it worse. An alternative would be to have the feature flag directly imply a bundle so only the one line is needed (this would depend on blis-src and f2c-netlib-lapack-src).

lapack-src = { version = "0.9", features = ["blis-f2c-netlib"] }

This is less flexible, especially with respect to system-provided BLAS-only. As a weird example, IBM's ESSL provides BLAS along with LAPACK symbol collisions that have a different API (eyeroll justified), so it's common for apps to link -llapack -lessl to prefer the standard (Netlib) LAPACK symbols while still using ESSL for BLAS. I don't really care about ESSL, but want a reliable model for BLAS and LAPACK to be provided from different sources.

[tbetcke]

Would a sensible choice be to use Flexiblas as generic interface and have everything else handled from there?

Also, when compiling just Lapack how can you make sure that the right calling convention for complex return arguments is used? I noticed a recent PR in the f2clapack part from PETSc. Something similar may presumably need to be done here?

[jedbrown]

It seems to me the benefit of FlexiBLAS is to delay choice of which implementation is used until runtime when linking shared libs. As such, it's great when packaged by distributions (and Rust can use system BLAS), but what is the benefit when linking statically? It takes a while to build and users just want their app to run.

Regarding complex return ABI, it'd be best if this could be queried (it needs to be known to call the Fortran symbols). It is indeed an issue with f2c'd code that it only supports one convention.