pip compilation issues

[jeremyagray]

I keep having a multiple definition error of several functions when using sundials with lapack and pycvodes trying to use headers in anyode. I created a branch to illustrate how I finally got it to compile and install (no errors/warnings) by passing in the appropriate enviroment variable (PYCVODES_USE_LAPACK='1'), but I wonder if there is a problem elsewhere with one of the libraries or something else. I didn't dive deeply enough into the source to see if this turned off all lapack use, but the tests in pycvodes/tests passed. I was intending to work on or at least document pip installation of chempy and got hung on pycvodes.

Hopefully all the relevant setup details are below; errors follow.

versions (current debian/bullseye):

• gcc/g++/gfortran 10.2.0 (deb)
• sundials 5.4.0 (source)
• cmake 3.18.4-1 (deb)
• libblas-dev 3.9.0-3 (deb)
• liblapack-dev 3.9.0-3 (deb)
• libboost-dev 1.71.0.3 (deb)
• libgsl-dev 2.6+dfsg-2 (deb)
• libsuitesparse-dev 1:5.8.1+dfsg-2 (deb)
• Python 3.8.6 (deb)

requirements.txt:

• apipkg==1.5
• attrs==20.2.0
• Cython==0.29.21
• execnet==1.7.1
• iniconfig==1.1.1
• numpy==1.19.3
• packaging==20.4
• pep8==1.7.1
• pluggy==0.13.1
• py==1.9.0
• pyflakes==2.2.0
• pyparsing==2.4.7
• pytest==6.1.2
• pytest-cache==1.0
• pytest-flakes==4.0.2
• pytest-pep8==1.0.6
• six==1.15.0
• toml==0.10.1

Build Errors:
python setup.py bdist_wheel

Compiling pycvodes/_cvodes.pyx because it changed.
[1/1] Cythonizing pycvodes/_cvodes.pyx
running bdist_wheel
running build
running build_py
running egg_info
writing pycvodes.egg-info/PKG-INFO
writing dependency_links to pycvodes.egg-info/dependency_links.txt
writing requirements to pycvodes.egg-info/requires.txt
writing top-level names to pycvodes.egg-info/top_level.txt
reading manifest file 'pycvodes.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pycvodes.egg-info/SOURCES.txt'
copying pycvodes/_cvodes.cpp -> build/lib.linux-x86_64-3.8/pycvodes
running build_ext
building 'pycvodes._cvodes' extension
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPYCVODES_NO_KLU=0 -DPYCVODES_NO_LAPACK=0 -DANYODE_NO_LAPACK=0 -I/home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include -Ipycvodes/include -Iexternal/anyode/include -I/home/gray/.virtualenvs/pycvodes/include -I/usr/include/python3.8 -c pycvodes/_cvodes.cpp -o build/temp.linux-x86_64-3.8/pycvodes/_cvodes.o -DVERSION_INFO="0.13.1" --std=c++11
In file included from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from pycvodes/_cvodes.cpp:667:
/home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:38:43: error: conflicting declaration of C function ‘void dgemv_(const char*, sunindextype*, sunindextype*, const double*, const double*, sunindextype*, const double*, sunindextype*, const double*, double*, sunindextype*)’
   38 | #define dgemv_f77       SUNDIALS_F77_FUNC(dgemv, DGEMV)
      |                                           ^~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:38:25: note: in expansion of macro ‘SUNDIALS_F77_FUNC’
   38 | #define dgemv_f77       SUNDIALS_F77_FUNC(dgemv, DGEMV)
      |                         ^~~~~~~~~~~~~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:122:13: note: in expansion of macro ‘dgemv_f77’
  122 | extern void dgemv_f77(const char *trans, sunindextype *m, sunindextype *n,
      |             ^~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:2:17: note: previous declaration ‘void dgemv_(const char*, int*, int*, const double*, const double*, int*, const double*, int*, const double*, double*, int*)’
    2 | extern "C" void dgemv_(const char* trans, int* m, int* n, const double* alpha, const double* a, int* lda,
      |                 ^~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:54:43: error: conflicting declaration of C function ‘void sgemv_(const char*, sunindextype*, sunindextype*, const float*, const float*, sunindextype*, const float*, sunindextype*, const float*, float*, sunindextype*)’
   54 | #define sgemv_f77       SUNDIALS_F77_FUNC(sgemv, SGEMV)
      |                                           ^~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:132:13: note: in expansion of macro ‘sgemv_f77’
  132 | extern void sgemv_f77(const char *trans, sunindextype *m, sunindextype *n,
      |             ^~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:4:17: note: previous declaration ‘void sgemv_(const char*, int*, int*, const float*, const float*, int*, const float*, int*, const float*, float*, int*)’
    4 | extern "C" void sgemv_(const char* trans, int* m, int* n, const float* alpha, const float* a, int* lda,
      |                 ^~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:42:43: error: conflicting declaration of C function ‘void dgbtrf_(const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, double*, sunindextype*, sunindextype*, sunindextype*)’
   42 | #define dgbtrf_f77      SUNDIALS_F77_FUNC(dgbtrf, DGBTRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:158:13: note: in expansion of macro ‘dgbtrf_f77’
  158 | extern void dgbtrf_f77(const sunindextype *m, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:25:17: note: previous declaration ‘void dgbtrf_(const int*, const int*, const int*, const int*, double*, int*, int*, int*)’
   25 | extern "C" void dgbtrf_(const int* dim1, const int* dim2, const int* kl, const int* ku, double* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:43:43: error: conflicting declaration of C function ‘void dgbtrs_(const char*, const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, double*, const sunindextype*, sunindextype*, double*, const sunindextype*, sunindextype*)’
   43 | #define dgbtrs_f77      SUNDIALS_F77_FUNC(dgbtrs, DGBTRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:163:13: note: in expansion of macro ‘dgbtrs_f77’
  163 | extern void dgbtrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:28:17: note: previous declaration ‘void dgbtrs_(const char*, const int*, const int*, const int*, const int*, double*, const int*, int*, double*, const int*, int*)’
   28 | extern "C" void dgbtrs_(const char* trans, const int* n, const int* kl, const int* ku, const int* nrhs, double* a,
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:44:43: error: conflicting declaration of C function ‘void dgetrf_(const sunindextype*, const sunindextype*, double*, sunindextype*, sunindextype*, sunindextype*)’
   44 | #define dgetrf_f77      SUNDIALS_F77_FUNC(dgetrf, DGETRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:179:13: note: in expansion of macro ‘dgetrf_f77’
  179 | extern void dgetrf_f77(const sunindextype *m, const sunindextype *n, double *a,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:14:17: note: previous declaration ‘void dgetrf_(const int*, const int*, double*, int*, int*, int*)’
   14 | extern "C" void dgetrf_(const int* dim1, const int* dim2, double* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:45:43: error: conflicting declaration of C function ‘void dgetrs_(const char*, const sunindextype*, const sunindextype*, double*, const sunindextype*, sunindextype*, double*, const sunindextype*, sunindextype*)’
   45 | #define dgetrs_f77      SUNDIALS_F77_FUNC(dgetrs, DGETRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:183:13: note: in expansion of macro ‘dgetrs_f77’
  183 | extern void dgetrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:17:17: note: previous declaration ‘void dgetrs_(const char*, const int*, const int*, double*, const int*, int*, double*, const int*, int*)’
   17 | extern "C" void dgetrs_(const char* trans, const int* n, const int* nrhs, double* a, const int* lda, int* ipiv, double * b, const int* ldb, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:58:43: error: conflicting declaration of C function ‘void sgbtrf_(const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, float*, sunindextype*, sunindextype*, sunindextype*)’
   58 | #define sgbtrf_f77      SUNDIALS_F77_FUNC(sgbtrf, SGBTRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:205:13: note: in expansion of macro ‘sgbtrf_f77’
  205 | extern void sgbtrf_f77(const sunindextype *m, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:26:17: note: previous declaration ‘void sgbtrf_(const int*, const int*, const int*, const int*, float*, int*, int*, int*)’
   26 | extern "C" void sgbtrf_(const int* dim1, const int* dim2, const int* kl, const int* ku, float* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:59:43: error: conflicting declaration of C function ‘void sgbtrs_(const char*, const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, float*, const sunindextype*, sunindextype*, float*, const sunindextype*, sunindextype*)’
   59 | #define sgbtrs_f77      SUNDIALS_F77_FUNC(sgbtrs, SGBTRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:210:13: note: in expansion of macro ‘sgbtrs_f77’
  210 | extern void sgbtrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:30:17: note: previous declaration ‘void sgbtrs_(const char*, const int*, const int*, const int*, const int*, float*, const int*, int*, float*, const int*, int*)’
   30 | extern "C" void sgbtrs_(const char* trans, const int* n, const int* kl, const int* ku, const int* nrhs, float* a,
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:60:43: error: conflicting declaration of C function ‘void sgetrf_(const sunindextype*, const sunindextype*, float*, sunindextype*, sunindextype*, sunindextype*)’
   60 | #define sgetrf_f77      SUNDIALS_F77_FUNC(sgetrf, SGETRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:226:13: note: in expansion of macro ‘sgetrf_f77’
  226 | extern void sgetrf_f77(const sunindextype *m, const sunindextype *n, float *a,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:15:17: note: previous declaration ‘void sgetrf_(const int*, const int*, float*, int*, int*, int*)’
   15 | extern "C" void sgetrf_(const int* dim1, const int* dim2, float* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:61:43: error: conflicting declaration of C function ‘void sgetrs_(const char*, const sunindextype*, const sunindextype*, float*, const sunindextype*, sunindextype*, float*, const sunindextype*, sunindextype*)’
   61 | #define sgetrs_f77      SUNDIALS_F77_FUNC(sgetrs, SGETRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:230:13: note: in expansion of macro ‘sgetrs_f77’
  230 | extern void sgetrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:18:17: note: previous declaration ‘void sgetrs_(const char*, const int*, const int*, float*, const int*, int*, float*, const int*, int*)’
   18 | extern "C" void sgetrs_(const char* trans, const int* n, const int* nrhs, float* a, const int* lda, int* ipiv, float * b, const int* ldb, int* info);
      |                 ^~~~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

[bjodah]

It's a mess, I know--believe me. Thank you for trying to document this.

Some things that come to mind here:

• I seem to remember that Sundials until recently expected one extra argument in the functions in LAPACK, compared to what is shipped with standard lapack. The way I've worked around this was to simply declare the function with this extra argument, and it seems (to me with my limited knowledge of Linux function calling convention ABI) that it works fine. But I think I've gotten rid of it (possibly excluding older versions of sundials). For some time I did test 3 or 4 different versions of sundials, with 3 different precisions, and double precision with and without lapack. But it was too much work due to the combinatorial nature of it.

• Did you compile sundials with 32 bit integer size? (sundials cmake config is tricky, at least it used to simply ignore LAPACK_ENABLE=ON unless SUNDIALS_INDEX_SIZE was set to 32...)

[bjodah]

What if you insert this line:

#ifndef _SUNDIALS_LAPACK_H

above this one?
https://github.com/bjodah/anyode/blob/ce9db8c91adf036657219a3cdf3c251483a3f0f7/include/anyode/anyode_blas_lapack.hpp#L2

(and insert a #endif after the relevant extern declarations?)

[jeremyagray]

I still get the same errors (no new ones). Compilations fails with PYCVODES_USE_LAPACK='0' or unset and compiles with it set to '1'.

…

On Sat, Oct 31, 2020 at 6:09 AM Bjorn ***@***.***> wrote: What if you insert this line: #ifndef _SUNDIALS_LAPACK_H above this one? https://github.com/bjodah/anyode/blob/ce9db8c91adf036657219a3cdf3c251483a3f0f7/include/anyode/anyode_blas_lapack.hpp#L2 (and insert a #endif after the relevant extern declarations?) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOQCHS332HKUD5FPHY5UNS3SNPV7FANCNFSM4TEID3FQ> .

-- Jeremy A. Gray

[bjodah]

sorry, I don't quite follow, did you export environment variables on the command line prior to installing?, i.e.:

export PYCVODES_NO_LAPACK=1 PYCVODES_NO_KLU=1
python3 -m pip install .

regarding requirements.txt, I think it's deprecated since 5 years or so? setuptools & requires arguments are preferred I believe? (way back, I used to maintain a requirements.txt in the repository even...)

[jeremyagray]

I added the #ifndef/#endif, then cleared the environment so that `PYCVODES_NO_LAPACK` was unset so that it would build as normal with `python setup.py build` in the pycvodes top directory, and I got the same multiple definition errors. Then, I exported and ran `export PYCVODES_NO_LAPACK='1' && python setup.py build` in the pycvodes top directory and successfully compiled. I do the same test with chempy (with and without `PYCVODES_NO_LAPACK` in the environment), and install always fails on pycvodes if `PYCVODES_NO_LAPACK='1'` is not set in the environment. I have not changed `PYCVODES_NO_KLU`. I'm using `requirements.txt` (in chempy) right now as a list to get a virtualenv set up for testing and development (deployment only, not packaging and installing chempy).

…

On Sat, Oct 31, 2020 at 10:50 AM Bjorn ***@***.***> wrote: sorry, I don't quite follow, did you export environment variables on the command line prior to installing?, i.e.: export PYCVODES_NO_LAPACK=1 PYCVODES_NO_KLU=1 python3 -m pip install . regarding requirements.txt, I think it's deprecated since 5 years or so? setuptools & requires arguments are preferred I believe? (way back, I used to maintain a requirements.txt in the repository even...) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOQCHS53WZ6TW4AXYMR4AJTSNQW6BANCNFSM4TEID3FQ> .

-- Jeremy A. Gray

[bjodah]

Alright, hard to say exactly why it is failing. Support for both with & without lapack was added to allow compilation of sundials with long double (which BLAS/LAPACK does not support). There's been a lot of back and forth in this area over the years, simply because each new environment has essentially shown to need its own treatment.

When one has multiple versions of sundials on the same machine, I have found that my sanity is best preserved by directly setting CFLAGS & LDFLAGS:

export CFLAGS="-isystem /opt/sundials-5.2.0-noklu-nolapack-extended-int64/include -DPYCVODES_NO_KLU=1 -DPYCVODES_NO_LAPACK=1"
export LDFLAGS="-Wl,--disable-new-dtags -Wl,-rpath,/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib -L/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib"

[jeremyagray]

Let me see if I understand everything. I intend to have just one sundials installation. I have lapack, blas, boost, suitesparse, gsl, and sundials either in the system include/ld paths or in a searched path, so all of them can include and link with each other as necessary (and I can adjust CFLAGS/LDFLAGS if necessary). So if sundials is compiled without klu and lapack, does pycvodes/anyode include/link with suitesparse and lapack directly? Is there any functionality lost that might be useful in pycvodes or chempy? I think my question and this issue rests on how sundials should be configured before compilation because `.drone.yml` indicates that sundials should be compiled without klu and lapack. Or alternatively, if sundials is compiled with certain features (klu or lapack), should pycvodes/anyode ignore parts of the sundials interface in deference to the implementation in pycvodes/anyode?

…

On Sat, Oct 31, 2020 at 11:44 AM Bjorn ***@***.***> wrote: Alright, hard to say exactly why it is failing. Supporting both with & without lapack done to allow compilation of sundials with long double (which BLAS/LAPACK does not support). There's been a lot of back and forth in this area over the years, simply because each new environment has essentially shown to need its own treatment. When one has multiple versions of sundials on the same machine, I have found that my sanity is best preserved by directly setting CFLAGS & LDFLAGS: export CFLAGS="-isystem /opt/sundials-5.2.0-noklu-nolapack-extended-int64/include -DPYCVODES_NO_KLU=1 -DPYCVODES_NO_LAPACK=1" export LDFLAGS="-Wl,--disable-new-dtags -Wl,-rpath,/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib -L/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib" — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOQCHS24DN6Q43SDJEQVILDSNQ5H5ANCNFSM4TEID3FQ> .

-- Jeremy A. Gray

[bjodah]

So if sundials is compiled without klu and
lapack, does pycvodes/anyode include/link with suitesparse and lapack
directly?

It should not.

Is there any functionality lost that might be useful in
pycvodes or chempy?

No, after some problem size LAPACK will be faster, but for small problems, I've seen the inverse, i.e. not using LAPACK is significantly faster.

pycvodes can exploit matrix sparseness when KLU is available. ChemPy does not use this however.

Right now I just upgraded the Debian version to Bullseye in the Dockerfile in ChemPy's repository:
bjodah/chempy#186

All tests pass in that environment (you need docker installed to run that script on your machine in case you want to compare).

[jeremyagray]

I've finally tracked down the source of the issue, but I don't know enough about C++ to completely understand what is happening. Since we tried an include guard with `#ifndef _SUNDIALS_LAPACK_H` and it did not work, I tried some other variations that also failed. After some research, I found that the include guards work against multiple inclusion but not multiple definition. Changing all the `extern "C"` in `anyode/include/anyode/anyode_blas_lapack.h` to just `extern` allowed pycvodes to build without error (tests ran successfully). I would assume this has something to do with sundials using `extern "C"` in `sundials_lapack.h`. This is on debian bullseye current, with sundials-5.5.0 compiled with ``` cmake -DCMAKE_INSTALL_PREFIX=/home/gray/.local -DLAPACK_ENABLE=ON -DKLU_ENABLE=ON -DKLU_INCLUDE_DIR=/usr/include/suitesparse -DKLU_LIBRARY_DIR=/usr/lib/x86_64-linux-gnu .. && make && make install ```

…

On Tue, Nov 24, 2020 at 3:05 AM Bjorn ***@***.***> wrote: So if sundials is compiled without klu and lapack, does pycvodes/anyode include/link with suitesparse and lapack directly? It should not. Is there any functionality lost that might be useful in pycvodes or chempy? No, after some problem size LAPACK will be faster, but for small problems, I've seen the inverse, i.e. not using LAPACK is *significantly* faster. pycvodes can exploit matrix sparseness when KLU is available. ChemPy does not use this however. Right now I just upgraded the Debian version to Bullseye in the Dockerfile in ChemPy's repository: bjodah/chempy#186 <bjodah/chempy#186> All tests pass in that environment (you need docker installed to run that script on your machine in case you want to compare). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOQCHS3KYTYIO75JSVTIERLSRNZMVANCNFSM4TEID3FQ> .

-- Jeremy A. Gray

[bjodah]

Oh, that's a tricky one.
Often extern "C" { itself is guarded by #ifdef __cplusplus (to allow compilation with C++ compilers as well as C compilers). Sounds like I may have forgotten such a guard somewhere then.