HIP build adds non-existent include path for hipfft

[kostrzewa]

In

https://github.com/lattice/quda/blame/914b462d802e515eb20b7313f2cd9d0b1264b8f4/lib/targets/hip/target_hip.cmake#L135

the path

target_include_directories(quda PUBLIC ${ROCM_PATH}/hipfft/include)

is added to the include directories.

While QUDA builds fine, configuring another code linking against QUDA using CMake will fail like so:

CMake Error in src/lib/CMakeLists.txt:
  Imported target "QUDA::quda" includes non-existent path

    "/opt/rocm-6.3.4/hipfft/include"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

This may be a ROCm version issue (6.3.4 and 6.4.4 are the latest we can use on LUMI-G) but at least for those two versions, the correct path is ${ROCM_PATH}/include/hipfft

[kostrzewa]

@dmcdougall is the path to the include file the other way around with newer ROCm versions?

[kostrzewa]

@kottnad I can thus confirm what you told me and the issue seems to be here.

[dmcdougall]

I'm confused by this. The header files were reorganised quite a long time ago.

[kostrzewa]

I'm confused by this. The header files were reorganised quite a long time ago.

Thanks for the quick reply!

This means that the "correct" ${ROCM_PATH}/include/hipfft could simply be used in QUDA likely without ill effects, right?

[dmcdougall]

This means that the "correct" ${ROCM_PATH}/include/hipfft could simply be used in QUDA likely without ill effects, right?

It depends what you mean by "ill effects". I'm sure someone somewhere is using an ancient ROCm and expects it to still work. But those configurations come with worse problems than a header file that now lives in a different location.

Yes, I highly encourage implementing your suggestion. What confuses me is that I would have expected to hear about issues like this pretty soon after the change happened...

[dmcdougall]

On a sort-of related note. Is there any reason we expose the rocfft headers to clients of QUDA? To first order, I would expect all that to be hidden away by the quda* interfaces.

[kostrzewa]

In the code in question in our CMakeLists.txt we do:

if(TM_USE_QUDA)                                                                                                                           
  find_package(QUDA REQUIRED config)    
endif()    

and then further down the line

target_link_libraries(
  tmlqcd
  PUBLIC # ...
 # ...
           $<$<BOOL:${TM_USE_QUDA}>:QUDA::quda>
)

which seems to pull in the hipfft header.

I guess there should be no problem if target_hip.cmake did: target_include_directories(quda PRIVATE ${ROCM_PATH}/include/hipff), right?

[kostrzewa]

Yes, I highly encourage implementing your suggestion. What confuses me is that I would have expected to hear about issues like this pretty soon after the change happened...

I wouldn't be surprised if other codes maybe have a legacy way of linking against QUDA (even with CMake). In tmLQCD we only noticed after moving our build system to CMake from our super-hacky "semi-autoconf" build.

[dmcdougall]

I guess there should be no problem if target_hip.cmake did: target_include_directories(quda PRIVATE ${ROCM_PATH}/include/hipff), right?

It depends.

It'll work if QUDA doesn't require that its client consume those headers.

Sometimes exposing a header like this is intentional; a dependency's datatype is exposed to the client.

Sometimes it is unintentional; someone accidentally dumped a #include for a header dependency in a publicly consumed header, rather than putting it only where it is needed in an internal header for internal consumption.

I don't know which one of these is true for QUDA. I assumed QUDA would want to hide hip/cuda/sycl implementation details away so that clients don't have to worry about them, but this is just an assumption.