conda-build: Build for Windows #2252

jjerphan · 2025-03-19T14:44:58Z

Reference Issues/PRs

Fix #2604.

What does this implement or fix?

Add setup for building conda packages for Windows before they are published on conda-forge.

Any other comments?

Checklist

Checklist for code changes...

Have you updated the relevant docstrings, documentation and copyright notice?
Is this contribution tested against all ArcticDB's features?
Do all exceptions introduced raise appropriate error messages?
Are API changes highlighted in the PR description?
Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

jjerphan · 2025-12-05T09:57:21Z

environment-dev.yml

-  - libprotobuf <7
+    # We have to use the build of libprotobuf starting from this PR:
+    # https://github.com/conda-forge/libprotobuf-feedstock/pull/277
+  - sel(win): libprotobuf==5.29.3=h*_3


This is a known installable and patched version of libprotobuf 5.

libprotobuf 6 is not as installable for Windows in my experience, and also would need to be patched (see conda-forge/libprotobuf-feedstock#275 (comment)).

Signed-off-by: Julien Jerphanion <[email protected]>

vasil-pashov · 2025-12-05T11:24:33Z

.github/workflows/build_with_conda.yml

+          set CMAKE_GENERATOR_PLATFORM=
+          set CMAKE_GENERATOR_TOOLSET=


I find it odd that this is needed only for the windows conda build. The other builds don't set this and still use ninja. How is conda different? What's the error that's raised if these are not added?

vasil-pashov · 2025-12-05T11:27:04Z

.github/workflows/build_with_conda.yml

+        # DONT use recursive submodules checkout to simulate conda feedstock build
+        # with:
+        #   submodules: recursive


nit: Why not just remove these?

Mainly to keep the style of this workflow file (this comment exists for Linux and Mac and is informative I think).

This file needs to be reformated and improved, and this better be done in another PR IMO.

.github/workflows/build_with_conda.yml

vasil-pashov · 2025-12-05T11:38:20Z

cpp/CMakeLists.txt

+    if(${ARCTICDB_USING_CONDA})
+        add_compile_definitions(
+            # Required to link against zstd.dll
+            # See: https://github.com/facebook/zstd/blob/dev/lib/zstd.h#L47-L53
+            ZSTD_DLL_IMPORT=1
+
+            # Required to link against lz4.dll
+            # See: https://github.com/lz4/lz4/blob/dev/lib/lz4frame.h#L79-L85
+            LZ4_DLL_IMPORT=1
+
+            # Required to link against protobuf.dll
+            # See: https://github.com/protocolbuffers/protobuf/blob/384fabf35a9b5d1f1502edaf9a139b1f51551a01/cmake/README.md?plain=1#L296-L319
+            PROTOBUF_USE_DLLS=1
+
+            # Required to link against fmt.dll
+            FMT_SHARED=1
+
+            # Required to link against spdlog.dll
+            # See: https://github.com/gabime/spdlog/blob/faa0a7a9c5a3550ed5461fab7d8e31c37fd1a2ef/include/spdlog/common.h#L28-L48
+            SPDLOG_COMPILED_LIB=1
+            SPDLOG_SHARED_LIB=1
+            SPDLOG_HEADER_ONLY=0
+
+            # Indicate that we are building using fmt's external DLL instead of spdlog's internal fmt
+            SPDLOG_FMT_EXTERNAL=1
+
+            # Required to link against azure-core.dll
+            # See: https://github.com/Azure/azure-sdk-for-cpp/blob/c914c0933fa4835c6618a42bb276d6077423b170/sdk/storage/azure-storage-common/inc/azure/storage/common/dll_import_export.hpp#L19
+            AZ_CORE_DLL=1
+
+            # Required to link against azure-identity.dll
+            # See: https://github.com/Azure/azure-sdk-for-cpp/blob/bf36db827f5ea1ef2e38953875c6c335b4242b2e/sdk/identity/azure-identity/inc/azure/identity/dll_import_export.hpp
+            AZ_IDENTITY_DLL=1
+
+            # Required to link against azure-storage-common.dll
+            # See: https://github.com/Azure/azure-sdk-for-cpp/blob/c914c0933fa4835c6618a42bb276d6077423b170/sdk/storage/azure-storage-common/inc/azure/storage/common/dll_import_export.hpp#L19
+            AZ_STORAGE_COMMON_DLL=1
+
+            # Required to link against azure-storage-blobs.dll
+            # See: https://github.com/Azure/azure-sdk-for-cpp/blob/bf36db827f5ea1ef2e38953875c6c335b4242b2e/sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/dll_import_export.hpp
+            AZ_STORAGE_BLOBS_DLL=1
+        )
+    endif()


I agree this doesn't look right. We can merge it like this, though I'd appreciate if you can investigate it at some point.

jjerphan

Replying to Vasil's review.

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan · 2025-12-05T17:51:12Z

@vasil-pashov: one of the thread disappeared with e89bc80 which gives the errors in the logs.

Here is the error for posterity:

[vcvarsall.bat] Environment initialized for: 'x64'
CMake Error at CMakeLists.txt:13 (project):
  Generator

    Ninja

  does not support platform specification, but platform

    x64

  was specified.


-- Configuring incomplete, errors occurred!
CMake Error:
  Generator

    Ninja

  does not support platform specification, but platform

    x64

  was specified.

Full logs

This thread was a hint to unset CMAKE_GENERATOR_{PLATFORM,TOOLSET}.

This reverts commit e89bc80.

Signed-off-by: Julien Jerphanion <[email protected]>

"[...] By contrast, PRE_TEST delays test discovery until just prior to test execution. This way test discovery occurs in the target environment where the test has a better chance at finding appropriate runtime dependencies." See: https://cmake.org/cmake/help/latest/module/GoogleTest.html Signed-off-by: Julien Jerphanion <[email protected]>

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan added the patch Small change, should increase patch version label Mar 19, 2025

jjerphan force-pushed the build/windows-conda-build branch 7 times, most recently from 9e226d0 to 0520427 Compare March 20, 2025 10:09

jjerphan changed the title ~~ci: Add workflow for Windows~~ conda-build: Add workflow for Windows Mar 21, 2025

jjerphan force-pushed the build/windows-conda-build branch 13 times, most recently from 34410d5 to 9323534 Compare March 24, 2025 15:59

jjerphan mentioned this pull request Mar 24, 2025

When building on Windows w/swiftc, can accidently include both msvc and gcc math.inls. awslabs/aws-c-common#1155

Closed

jjerphan force-pushed the build/windows-conda-build branch 7 times, most recently from f9ddf0d to 68c4487 Compare March 25, 2025 17:23

jjerphan commented Dec 5, 2025

View reviewed changes

jjerphan added the minor Feature change, should increase minor version label Dec 5, 2025

jjerphan force-pushed the build/windows-conda-build branch 2 times, most recently from 8fc8184 to dee474d Compare December 5, 2025 10:36

alexowens90 approved these changes Dec 5, 2025

View reviewed changes

jjerphan added 5 commits December 5, 2025 12:03

ci: Add GitHub workflow for Windows conda-build

0ce9aca

Signed-off-by: Julien Jerphanion <[email protected]>

Adapt calling convention for ternary_tranform's lambda

a00335d

Signed-off-by: Julien Jerphanion <[email protected]>

Adapt conda environment for Windows

22c1062

Signed-off-by: Julien Jerphanion <[email protected]>

Adapt Windows' presets

d045c31

Signed-off-by: Julien Jerphanion <[email protected]>

Adapt CMakelLists.txt for Windows conda build

10b2105

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan force-pushed the build/windows-conda-build branch from dee474d to 10b2105 Compare December 5, 2025 11:04

vasil-pashov reviewed Dec 5, 2025

View reviewed changes

jjerphan commented Dec 5, 2025

View reviewed changes

jjerphan added 2 commits December 5, 2025 18:34

Reintroduce environment variables for building C++ tests

a7f041a

Signed-off-by: Julien Jerphanion <[email protected]>

Remove explicit build instructions

975dc61

jjerphan force-pushed the build/windows-conda-build branch from 38bbf3f to 975dc61 Compare December 5, 2025 17:35

Do not change CMAKE_GENERATOR_{PLATFORM,TOOLSET}

e89bc80

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan added 2 commits December 5, 2025 18:52

Revert "Do not change CMAKE_GENERATOR_{PLATFORM,TOOLSET}"

045464c

This reverts commit e89bc80.

Set CTEST_OUTPUT_ON_FAILURE as part of env variables

88fac60

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan force-pushed the build/windows-conda-build branch from 7a91a93 to 88fac60 Compare December 8, 2025 08:19

Split in different steps

aa3f369

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan force-pushed the build/windows-conda-build branch from 0917fb4 to aa3f369 Compare December 8, 2025 08:52

jjerphan added 6 commits December 8, 2025 10:16

Test workaround for rapidcheck

9e85886

Signed-off-by: Julien Jerphanion <[email protected]>

Explicitly link Python so TARGET_RUNTIME_DLLS can detect Python DLLs

e84681c

Signed-off-by: Julien Jerphanion <[email protected]>

Merge two steps

6f7993d

Signed-off-by: Julien Jerphanion <[email protected]>

Do not build rapidcheck_gtest

e5c40f2

Signed-off-by: Julien Jerphanion <[email protected]>

Do not build and run C++ tests on Windows

758ba2a

Signed-off-by: Julien Jerphanion <[email protected]>

jjerphan force-pushed the build/windows-conda-build branch from 522d8e0 to 758ba2a Compare December 8, 2025 10:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

conda-build: Build for Windows #2252

conda-build: Build for Windows #2252

jjerphan commented Mar 19, 2025 •

edited

Loading

Uh oh!

jjerphan Dec 5, 2025

Uh oh!

vasil-pashov Dec 5, 2025

Uh oh!

vasil-pashov Dec 5, 2025

Uh oh!

jjerphan Dec 5, 2025

Uh oh!

Uh oh!

vasil-pashov Dec 5, 2025

Uh oh!

jjerphan left a comment

Uh oh!

jjerphan commented Dec 5, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

conda-build: Build for Windows #2252

Are you sure you want to change the base?

conda-build: Build for Windows #2252

Conversation

jjerphan commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement or fix?

Any other comments?

Checklist

Uh oh!

jjerphan Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

vasil-pashov Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

vasil-pashov Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

jjerphan Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vasil-pashov Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

jjerphan left a comment

Choose a reason for hiding this comment

Uh oh!

jjerphan commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jjerphan commented Mar 19, 2025 •

edited

Loading

jjerphan commented Dec 5, 2025 •

edited

Loading