Windows builder in CI

[michaelkirk]

I'm not very familiar with windows builds, but we have a steady stream of support requests about it.

I think it could be helpful to have an official build in CI to point people to, and also to have some confidence that it's possible to build this crate on windows at all.

Is anyone out there interested in setting one up?

related: #79, #156, #158

[cerrussell]

@michaelkirk I'll take a stab at it.

[michaelkirk]

Awesome! Just in case you weren't already aware, our existing CI runs are configured at: https://github.com/georust/proj/blob/main/.github/workflows/test.yml

I'm imaging we can add one or more entries for windows there, like we have for macos and ubuntu.

[cerrussell]

@michaelkirk So I've run into an issue related to curl and the (failure) to link libraries. At first, I thought this was something with sqlite3 because that's what cargo reports. However, I eventually identified that curl is the source. Please see this commit over at OSGEO/PROJ. They enabled linking to a static library because otherwise, Windows requires a whole bunch of other network libraries to be linked. Unfortunately for this project, that solution is no good because networkfilemanager.cpp.obj is calling functions exclusive to the dynamic library (e.g. __imp_curl_easy_cleanup etc.).

So I guess the obvious, if not necessarily feasible, options are to either go down the rabbit hole with the extra network libraries or else to provide alternative methods for linking that don't rely on the dynamic curl library.

[cerrussell]

I'm going to try grabbing the stuff that isn't being linked manually. Sounds like that might work per #158

[urschrei]

@cerrussell Just to check – we've swapped out cURL for Rust-native network libraries and we disable linking to it when building specifically to avoid the dependency on macOS / Linux / BSD…I can't see your progress, but I just wanted to ensure we aren't working at cross-purposes – is cURL necessary elsewhere in the Windows build process?

[cerrussell]

@urschrei Thanks for your input, that's really helpful. I think what happened on my side is that initially cargo build would fail and say curl is required for libproj, and libproj is a required component of proj so I kind of took that at face value and ran in the wrong direction.

I don't think curl is used apart from linking, so I am optimistic that I'll ultimately get it working.

[cerrussell]

So just an update @urschrei @michaelkirk

I've managed to set up windows builds using MSYS2. MSYS2 offers the significant benefit of being able to get all the dependencies from one of their repos and this was a much better experience than trying to cobble everything together using a mixture of vcpkg, chocolatey, and building from source. It was also much faster than when I was using an MSVC toolchain.

Currently, I have only proj-windows set up and it uses an MSYS2 provided copy of proj for linking. You can see that here. Note that there are 12 different windows builds configured - this is certainly overkill but I was curious to see how they compared in terms of build times. Clang64 appears to be noticeably faster.

I haven't had any luck as of yet building libproj from source, but I wonder if that's truly a necessity for Windows when it can be so easily installed from the MSYS2 repo. If there is an existing proj installation, the path would just need to be exported to the environment path so it can be found.

[temeddix]

I should say that it's extremely hard to use this crate on Windows right now, since we have to install pkg-config, libproj, otherwise sqlite3, which doesn't even have an installer...

I extracted sqlite3 in C:\sqlite and this still this crate won't compile. I have bundled_proj feature enabled, added sqlite3.exe's directory to PATH, etc...

   Compiling proj-sys v0.23.2
error: failed to run custom build command for `proj-sys v0.23.2`         

Caused by:
  process didn't exit successfully: `C:\Users\usern\Documents\GitHub\api\target\release\build\proj-sys-7e9234b83144766b\build-script-build` (exit code: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-msvc = None
  CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_msvc = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-pc-windows-msvc = None
  CMAKE_GENERATOR_x86_64_pc_windows_msvc = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-pc-windows-msvc = None
  CMAKE_PREFIX_PATH_x86_64_pc_windows_msvc = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-pc-windows-msvc = None
  CMAKE_x86_64_pc_windows_msvc = None
  HOST_CMAKE = None
  CMAKE = None
  running: "cmake" "C:\\Users\\usern\\.cargo\\registry\\src\\index.crates.io-6f17d22bba15001f\\proj-sys-0.23.2\\PROJSRC/proj/proj-9.2.1" "-G" "Visual Studio 17 2022" "-Thost=x64" "-Ax64" "-DBUILD_SHARED_LIBS=OFF" "-DBUILD_TESTING=OFF" "-DBUILD_CCT=OFF" "-DBUILD_CS2CS=OFF" "-DBUILD_GEOD=OFF" "-DBUILD_GIE=OFF" "-DBUILD_PROJ=OFF" "-DBUILD_PROJINFO=OFF" "-DBUILD_PROJSYNC=OFF" "-DENABLE_CURL=OFF" "-DENABLE_TIFF=OFF" "-DCMAKE_INSTALL_PREFIX=C:\\Users\\usern\\Documents\\GitHub\\api\\target\\release\\build\\proj-sys-ae234ba95e186ddf\\out" "-DCMAKE_C_FLAGS= -nologo -MD -Brepro" "-DCMAKE_C_FLAGS_RELEASE= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS_RELEASE= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS_RELEASE= -nologo -MD -Brepro" "-DCMAKE_BUILD_TYPE=Release"
  -- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.22621.
  -- Requiring C++11
  -- Requiring C++11 - done
  -- Requiring C99
  -- Requiring C99 - done
  -- Configuring PROJ:
  -- PROJ_VERSION                   = 9.2.1
  -- PROJ_ABI_VERSION               = 9_2
  -- nlohmann/json: internal
  -- Testing disabled
  -- Configuring proj library:
  -- ENABLE_IPO                     = OFF
  -- PROJ_CORE_TARGET_OUTPUT_NAME   = proj_9_2
  -- BUILD_SHARED_LIBS              = OFF
  -- PROJ_LIBRARIES                 = proj
  -- Configuring incomplete, errors occurred!

  --- stderr
  feature flags specified source build
  building libproj from source
  disabling tiff support
  CMake Deprecation Warning at cmake/Ccache.cmake:10 (cmake_minimum_required):
    Compatibility with CMake < 3.5 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.
  Call Stack (most recent call first):
    CMakeLists.txt:126 (include)


  CMake Error at CMakeLists.txt:181 (message):
    sqlite3 dependency not found!


  CMake Error at CMakeLists.txt:187 (message):
    sqlite3 >= 3.11 required!


  CMake Warning at CMakeLists.txt:206 (message):
    TIFF support is not enabled and will result in the inability to read some
    grids


  thread 'main' panicked at '
  command did not execute successfully, got: exit code: 1

[urschrei]

@temeddix There's nothing we can do about third-party dependencies and their availability on a given platform. If you want a better Windows experience, feel free to contribute patches. As open source developers our time is limited and we rely on contributions from interested users who feel that support for a feature or platform is lacking.

[temeddix]

At least I think I can help on improving docs on how to install things on Windows, I'll try to make a PR on that if I succeed

[cerrussell]

@temeddix Definitely try using msys2 on Windows if possible... so much easier.

[temeddix]

Thanks :) well, I'm trying to develop code on windows and deploy it to Ubuntu webserver....so I need to make this work on both of them.