[build] debug broken?

[simogasp]

It seems that the debug build is broken for cuda >= 8

On cuda 8, a static debug build fails (ubuntu 14 on travis):

-- Generating /home/travis/build/alicevision/popsift/build_debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_debug_macros.cu.o
/usr/local/cuda/bin/nvcc /home/travis/build/alicevision/popsift/src/popsift/common/debug_macros.cu -dc -o /home/travis/build/alicevision/popsift/build_debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_debug_macros.cu.o -ccbin /usr/bin/gcc -m64 --std c++11 -Xcompiler ,\"-g\" -Xcompiler -rdynamic -lineinfo --keep --source-in-ptx -Xcompiler -fPIC -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_62,code=sm_62 -gencode arch=compute_62,code=compute_62 --default-stream legacy -G -DNVCC -I/usr/include -I/home/travis/build/alicevision/popsift/build_debug/src/popsift -I/usr/local/cuda/include -I/usr/local/cuda/include
Generated /home/travis/build/alicevision/popsift/build_debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_debug_macros.cu.o successfully.
ptxas error   : Entry function '_ZN6thrust6system4cuda6detail5bulk_6detail15launch_by_valueILj512ENS4_9cuda_taskINS3_14parallel_groupINS3_16concurrent_groupINS3_5agentILm3EEELm512EEELm0EEENS4_7closureINS2_11scan_detail16inclusive_scan_nENS_5tupleINS4_6cursorILj1EEENS_6detail15normal_iteratorINS_7pointerIjNS2_3tagENS_11use_defaultESN_EEEElSP_NS_4plusIjEENS_9null_typeESS_SS_SS_SS_EEEEEEEEvT0_' with max regcount of 64 calls function '_ZN6thrust6system4cuda6detail5bulk_6detail11scan_detail16scan_with_bufferILb1ELm512ELm3ENS_6detail15normal_iteratorINS_7pointerIjNS2_3tagENS_11use_defaultESB_EEEESD_jNS_4plusIjEEEEvRNS3_16concurrent_groupINS3_5agentIXT1_EEEXT0_EEET2_SL_T3_T4_T5_RNS5_11scan_bufferIXT0_EXT1_ESL_SM_SO_EE' with regcount of 78
ptxas error   : Entry function '_ZN6thrust6system4cuda6detail5bulk_6detail15launch_by_valueILj512ENS4_9cuda_taskINS3_14parallel_groupINS3_16concurrent_groupINS3_5agentILm3EEELm512EEELm0EEENS4_7closureINS2_11scan_detail16inclusive_scan_nENS_5tupleINS4_6cursorILj1EEENS_6detail15normal_iteratorINS_7pointerIlNS2_3tagENS_11use_defaultESN_EEEElSP_NS_4plusIlEENS_9null_typeESS_SS_SS_SS_EEEEEEEEvT0_' with max regcount of 64 calls function '_ZN6thrust6system4cuda6detail5bulk_6detail11scan_detail16scan_with_bufferILb1ELm512ELm3ENS_6detail15normal_iteratorINS_7pointerIlNS2_3tagENS_11use_defaultESB_EEEESD_lNS_4plusIlEEEEvRNS3_16concurrent_groupINS3_5agentIXT1_EEEXT0_EEET2_SL_T3_T4_T5_RNS5_11scan_bufferIXT0_EXT1_ESL_SM_SO_EE' with regcount of 80
ptxas error   : Entry function 

On a cuda 10.2 dynamic debug build I get (ubuntu 18.04):

[  2%] Building NVCC intermediate link file src/CMakeFiles/popsift.dir/popsift_intermediate_link.o
cd popsift/cmake-build-debug/src && /usr/local/cuda/bin/nvcc -Xcompiler -rdynamic -lineinfo --keep --source-in-ptx -Xcompiler -fPIC -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_62,code=sm_62 -gencode arch=compute_62,code=compute_62 --default-stream legacy -G -m64 -ccbin /usr/bin/cc -dlink popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_features.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_constants.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_conf.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_gauss_filter.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_image.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_pyramid.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_octave.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_pyramid_build.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_pyramid_build_aa.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_pyramid_build_ai.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_pyramid_build_ra.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_pyramid_fixed.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_extremum.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_extrema.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_orientation.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_sift_desc.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_desc_loop.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_desc_iloop.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_desc_grid.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_desc_igrid.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_desc_notile.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_assist.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_plane_2d.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_write_plane_2d.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_debug_macros.cu.o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/common/./popsift_generated_device_prop.cu.o -o popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/./popsift_intermediate_link.o
nvcc warning : '--device-debug (-G)' overrides '--generate-line-info (-lineinfo)'
nvlink error   : Undefined reference to 'cudaOccupancyMaxActiveBlocksPerMultiprocessorWithFlags' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaPeekAtLastError' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaGetParameterBuffer' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaDeviceSynchronize' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaLaunchDevice' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaGetErrorString' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaGetLastError' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaGetDevice' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
nvlink error   : Undefined reference to 'cudaDeviceGetAttribute' in 'popsift/cmake-build-debug/src/CMakeFiles/popsift.dir/popsift/./popsift_generated_s_filtergrid.cu.o' (target: sm_62)
src/CMakeFiles/popsift.dir/build.make:8453: recipe for target 'src/CMakeFiles/popsift.dir/popsift_intermediate_link.o' failed

For the lattert seems like a linking missing but the only difference between release and debug command line is just the -G and -O3. And the calls to those functions are generated in both build types.

Any clue? @griwodz

[griwodz]

The 62 must be removed from the PopSift_CUDA_CC_LIST_BASIC. 62 is a Compute Capability for the single-board-computers Tegra and Jetson whose CPUs are ARM. The library that contains the CUDA functions exists only for ARM CPUs and it is therefore impossible to link to it on an x86 CPU.
The correct fix would probably be to make PopSift_CUDA_CC_LIST_BASIC dependent on CPU architecture as well as dependent on CUDA SDK version.

[simogasp]

Ok, according to this
https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/
also 35 must go. And indeed without these two the linking error disappear.

Maybe we can use his scheme to determine the CCs based on cuda version (removing the ones that are not supported by popsift).
We can also rely on the function provide by findCuda CUDA_SELECT_NVCC_ARCH_FLAGS
https://android.googlesource.com/platform/external/cmake/+/refs/heads/upstream-release/Modules/FindCUDA/select_compute_arch.cmake
which is quite similar to what we want to do. (keeping in mind that FindCuda is deprecated but not all the functionality are fully integrated natively). What do you think?

Also removing those two even the first error disappear 🎉
https://travis-ci.org/alicevision/popsift/jobs/654795951

[griwodz]

I think that we should attempt to write a CC selector in a separate file ChooseCudaCC.cmake that considers both minimal CC and host architecture. It could be used in AliceVision as well I'm going to make a PR in PopSift with a first proposal.

The CC 3.5 was used by several mobile cards like 740M and 750M. I don't know if it matters much, but it has never given me any trouble.

[griwodz]

Started the function in PR #75

[simogasp]

Ok sorry my bad, when i tried to remove just 62 it was still failing but maybe it forgot to clean. It works for 35 even on travis
https://travis-ci.org/alicevision/popsift/jobs/654880074

[simogasp]

fixed by #75

[gloria111]

seems the debug build is broken on vs:

[simogasp]

What cuda version are you using?
We probably need to update the arch flag for the newer versions of cuda

[gloria111]

the CUDA version is 11.6
NVIDIA GPU Computing Toolkit\CUDA\v11.6