CUDA incorrectly parsed and invalid options forwarded by cmake

[fmigneault]

I am trying to build the C++ API on Windows 10 with CUDA 10.1 for Visual Studio 2019.

Part of cmake seems to indicate that it uses CUDA 10.2, then counter indicates it uses CUDA 10.1
(I am targeting 10.1, all my env is configured to point to it)
Output from cmake:

  Environment variable CUDA_ROOT is set to:

    %DEVELOP%/cuda/cuda-10.1

  For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):
  E:/develop/torch/torch-1.5.0a-source-20200208/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  E:/develop/torch/torch-1.5.0a-source-20200208/share/cmake/Torch/TorchConfig.cmake:40 (find_package)
  CMakeLists.txt:13 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

Caffe2: CUDA detected: 10.2
Caffe2: CUDA nvcc is: E:/develop/cuda/cuda-10.1/bin/nvcc.exe
Caffe2: CUDA toolkit directory: E:/develop/cuda/cuda-10.1
Caffe2: Header version is: 10.1
CMake Warning (dev) at E:/develop/torch/torch-1.5.0a-source-20200208/share/cmake/Caffe2/public/cuda.cmake:106 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  CMake variable CUDNN_ROOT is set to:

    E:/develop/cuda/cudnn-10.1

  Environment variable CUDNN_ROOT is set to:

    %DEVELOP%/cuda/cuda-10.1/../cudnn-10.1

Resulting Visual Studio 2019 solution doesn't have the proper options forwarded :
(for example, I should have compute_61;sm_61)

And although cmake seemed to indicate it correctly found my CUDA 10.1 in the end, it still points to CUDA 10.2 when trying to compile:

I have managed to compile https://github.com/pytorch/pytorch without issue using CUDA 10.1, so I really think this is not a ENV/config issue of CUDA, but a cmake parsing problem.

[fmigneault]

For anyone who would encounter a similar issue, I have found that, although the employed enable_language(CUDA) is the new official method to find CUDA dependencies, it always failed detecting the right installation.
Simply using the deprecated find_package(CUDA) method worked, as it has always been...

[peterjc123]

Yes, we used find_package(CUDA) in PyTorch. It may need some other configuration to make it work.

[fmassa]

@fmigneault we have merged #1850, does this fix your issues?

[peterjc123]

@fmassa Actually this is not fixed in #1850.

[fmassa]

@peterjc123 so I suppose we still need #1866?

@fmigneault can you rebase your PR on top of master?

[bmanga]

@fmassa I think #1866 was also trying to fix #1845, so that PR can be closed. This issue seems unrelated.

[bmanga]

even with find_package(CUDA), I am still getting the linker error due to cudart_static.lib being looked for in v10.1 path instead of v10.2 . I am starting to think that this is pytorch's fault.

[fmigneault]

I have tried with both CUDA 10.1 and 10.2.
The correct static lib seems to be found and used in both cases on my side, as long as find_package is used and not enable_language. The later doesn't work even when explicitly providing CUDA_ROOT.

[bmanga]

Not sure what the problem could be on my end then.
Could you try specifying CMAKE_CUDA_COMPILER to point to the nvcc from v10.1 and see if that solves it for you?

[fmigneault]

Seems like there might be something fishy going on after all.
After compiling torch & vision both with CUDA 10.2, my sample app (also using it) attempts to find cudart64_101.dll.

[fmigneault]

No doesn't work with CMAKE_CUDA_COMPILER

CMake Error at C:/Program Files/CMake/share/cmake-3.16/Modules/CMakeTestCUDACompiler.cmake:46 (message):
  The CUDA compiler

    "E:/develop/cuda/v10.1/bin/nvcc.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: E:/develop/torch/vision-v0.6.0a-torch-1.5.0a-source-cuda-10.1/build/debug/CMakeFiles/CMakeTmp
    
    Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/MSBuild/Current/Bin/MSBuild.exe cmTC_d623a.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:m && Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Framework

    Copyright (C) Microsoft Corporation. All rights reserved.

    C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 10.2.targets(587,9): error : The CUDA Toolkit v10.2 directory '' does not exist.  Please verify the CUDA Toolkit is installed properly or define the CudaToolkitDir property to resolve this error. [E:\develop\torch\vision-v0.6.0a-torch-1.5.0a-source-cuda-10.1\build\debug\CMakeFiles\CMakeTmp\cmTC_d623a.vcxproj]
  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:9 (enable_language)