Windows CI workflow by TravisCardwell · Pull Request #29 · well-typed/libclang-bindings

TravisCardwell · 2026-02-09T06:12:17Z

Experimenting and debugging...

TravisCardwell · 2026-02-09T07:01:21Z

PSA: Do not try to use vim via tmate on Windows!

Doing so completely borks the session. Nothing I tried was able to exit or kill it. The only way out I found was to cancel the workflow from the web UI (and rescue the terminal using ENTER ~ .).

TravisCardwell · 2026-02-09T07:20:25Z

ed is not installed!?!? I am weeping POSIX tears.

# cat << EOF > check.c
#include <clang-c/Index.h>
#include <stdio.h>

int main(void) {
  CXString cxstring = clang_getClangVersion();
  printf("%s\n", clang_getCString(cxstring));
  clang_disposeString(cxstring);
  return 0;
}
EOF
# clang -I $($LLVM_CONFIG --includedir) -lclang -L $($LLVM_CONFIG --libdir) check.c
# ./a.exe
clang version 14.0.6

TravisCardwell · 2026-02-09T07:40:11Z

I tried building in a tmate session, and it reported the correct LLVM/Clang version! This was confirmed using ghci:

ghci> import Clang.Version
ghci> print clangVersionCompileTime
"clang version 14.0.6"
ghci> print clangVersion
ClangVersion (14,0,6)

When I exited the session, however, the same exact commands were run via CI and got the wrong version.

checking libclang version... clang version 20.1.8

TravisCardwell · 2026-02-09T09:52:07Z

BTW, I confirmed the obvious:

$LLVM_PATH is set correctly.
$LLVM_CONFIG is set correctly, and $LLVM_CONFIG --version looks good.
which llvm-config reports the same as $LLVM_CONFIG (since $PATH is configured).
$LLVM_CONFIG --libdir looks good.
$LLVM_CONFIG --includedir looks good.

I just ran a new session, breaking after setup. I ran cabal build all, noting that the reported LLVM/Clang version is still correct when done manually. I then exited the session to let CI continue. The cabal build all --only-dependencies step reported "Up to date" as expected, but the cabal build all did not.

Build profile: -w ghc-9.4.8 -O1
In order, the following will be built (use -v for more details):
 - libclang-bindings-0.1.0 (lib) (configuration changed)
 - libclang-bindings-0.1.0 (exe:clang-bootstrap) (configuration changed)
 - libclang-bindings-0.1.0 (test:test-clang-bindings) (configuration changed)
 - libclang-bindings-0.1.0 (test:clang-tutorial) (configuration changed)

I expected that to report "Up do date" as well and do not know how the configuration changed. Trying again, manually running cabal build all twice in a row results in "Up to date" as expected. After exiting the session, I see the same CI behavior, however: cabal build all is rebuilding (and linking incorrectly).

Specifying -v does not provide information about how the configuration changed.
Build profile -w ghc-9.4.8 -O1 is used in all cases.

Next steps:

Increase the verbosity and search for clues
Confirm that the $PATH is the same

TravisCardwell · 2026-02-09T21:39:04Z

Regarding the non-informative "configuration changed" message, I found the following Cabal issue: "configuration changed" is way too vague

It seems that mysterious "configuration changed" rebuilds are not uncommon on Windows.

One comment suggests that -v3 may provide clues as to what changed, but my tests did not provide any clues.

TravisCardwell · 2026-02-10T00:05:47Z

I confirmed that the $PATH in the tmate shell is indeed different than the $PATH in the other jobs. That is not surprising because it is an "interactive shell." Perhaps that is the reason why there is a "configuration changed" rebuild, and the differences in the $PATH might indicate why it is linking correctly in the shell but not in the job.

BTW, the $PATH is long. Even though echo "${PATH}" shows the full string in the [Debug] LLVM/Clang step, the same command was truncated (at 1024 bytes) in the Build step! (Why is there a discrepancy!?!?) I am therefore using echo "${PATH}" | tr ':' '\n' so that it prints one directory per line.

I am unable to pass commands via SSH so that they run without the interactive shell. I bet that this is deliberate, for security reasons, so I did not try to circumvent this restriction.

Here is a side-by-side diff with blocks of common lines omitted:

Jobs	`tmate`
`/mingw64/bin`	`/mingw64/bin`
	`/usr/local/bin`
`/usr/bin`	`/usr/bin`
`/c/Users/runneradmin/bin`	`/bin`
...	...
`/bin`	`/c/Program Files/Git/bin`
...	...
`/cmd`	`/c/Program Files/Git/cmd`
`/mingw64/bin`	`/c/Program Files/Git/mingw64/bin`
`/usr/bin`	`/c/Program Files/Git/usr/bin`
...	...
	`/usr/bin/site_perl`
	`/usr/bin/vendor_perl`
	`/usr/bin/core_perl`

I wrote a Bash script to check directories in the $PATH for certain conditions, which can be run in CI as well as in a tmate shell.

Job output:

noexist /c/Users/runneradmin/bin
llvm*   /c/tools/ghc-9.4.8/mingw/bin
clang*  /c/tools/ghc-9.4.8/mingw/bin
noexist /c/cabal/bin
noexist /c/Program Files (x86)/GitHub CLI
noexist /c/Program Files/Microsoft SQL Server/140/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/150/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/160/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/170/DTS/Binn
empty   /c/Strawberry/perl/site/bin
llvm*   /c/Program Files/LLVM/bin
clang*  /c/Program Files/LLVM/bin
noexist /c/Program Files (x86)/LLVM/bin

Shell output:

empty   /mingw64/bin
noexist /usr/local/bin
llvm*   /c/tools/ghc-9.4.8/mingw/bin
clang*  /c/tools/ghc-9.4.8/mingw/bin
noexist /c/cabal/bin
noexist /c/Program Files (x86)/GitHub CLI
noexist /c/Program Files/Microsoft SQL Server/140/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/150/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/160/DTS/Binn
noexist /c/Program Files/Microsoft SQL Server/170/DTS/Binn
empty   /c/Strawberry/perl/site/bin
llvm*   /c/Program Files/LLVM/bin
clang*  /c/Program Files/LLVM/bin
noexist /c/Program Files (x86)/LLVM/bin
empty   /usr/bin/site_perl
empty   /usr/bin/vendor_perl

Notes:

In both cases, LLVM/Clang commands are only in /c/tools/ghc-9.4.8/mingw/bin (the ones we want to use) and /c/Program Files/LLVM/bin (the ones that are pre-installed and we do not want to use). In both cases, /c/tools/ghc-9.4.8/mingw/bin comes first, as desired.
/mingw64/bin is not empty when jobs execute, but it is empty when using the shell. It does not contain LLVM/Clang commands, however.
/usr/local/bin is added to the path in the shell, even though it does not exist. It is not in the job path at all.
/c/Users/runneradmin/bin is in the job path but not the shell path. This directory does not exist either.
Some directories in the job path show up in the shell with a /c/Program Files/Git prefix. A few of them in the job path (without the prefix) are duplicates, so perhaps the job path is getting those wrong. None of them contain LLVM/Clang commands, however.
Directory /c/cabal/bin does not exist. This may indicate an issue with the Haskell setup implementation, but I shall not worry about this now.
Those Perl directories that are appended to the shell path are dubious, but they indeed exist. The filesystem is just a mess.
I am not sure about case-sensitivity in MinGW, but the script output matches expectations, so I have not searched for CLANG*, etc.

The $PATH differences do not explain why we get different behavior in the shell and the job.

TravisCardwell · 2026-02-10T00:43:11Z

The test executable can be checked with ldd.

Shell:

# find dist-newstyle -type f -name test-clang-bindings.exe | xargs ldd | grep clang
        libclang.dll => /c/tools/ghc-9.4.8/mingw/bin/libclang.dll (0x7ffb930c0000)

Job:

libclang.dll => /c/tools/ghc-9.4.8/mingw/bin/libclang.dll (0x7ffa54030000)

What!?!? This indicates that the test executable is linking to the correct DLL. In the same test run, however, I see the following output:

test-clang-bindings
  Test.Version
    current:            OK
      "clang version 20.1.8" parsed as (20,1,8)

Looking at the test code, I confirmed that the value that is parsed is indeed acquired using clang_getClangVersion, so it should be the value from libclang.dll.

The test executable also has the following link, which looks good:

libLLVM-14.dll => /c/tools/ghc-9.4.8/mingw/bin/libLLVM-14.dll (0x7ffa4fbb0000)

Surely /c/tools/ghc-9.4.8/mingw/bin/libclang.dll is linked with the correct LLVM DLL... I downloaded the tarball so that I can investigate on Linux. Using objdump, I confirmed that libclang.dll links to libLLVM-14.dll in the import table. That is a versioned link, so it should be fine.

libclang.dll also has dynamic links to the following DLLs, none of which raise any flags:

KERNEL32.dll
libc++.dll
api-ms-win-crt-environment-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-private-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-time-l1-1-0.dll
api-ms-win-crt-filesystem-l1-1-0.dll
VERSION.dll (a Windows system DLL notorious for issues and hacking)
ADVAPI32.dll
api-ms-win-crt-locale-l1-1-0.dll

TravisCardwell · 2026-02-16T08:04:27Z

I (finally) figured out what was going wrong! We must only ever add Windows-style directories to the PATH, and only rely on automatic translation from Windows-style to POSIX-style.

I had been adding a POSIX-style directory to the PATH. All of our steps were using Bash, where all paths are shown POSIX-style, and the PATH always looked fine. Everything worked fine When using tmate, but CI steps must run differently (even when using Bash), and the POSIX-style paths were not recognized when executing programs.

The directory that we add is computed, as we have different directories depending on if GHC was installed via Chocolatey or via GHCup, which is sometimes even determined by network conditions. I found that program cygpath is used to convert between directory formats, using cygpath -w to convert from POSIX-style to Windows-style. We can therefore continue to use Bash.

With this fixed, the correct libclang.dll is now found in the CI job!

TravisCardwell · 2026-02-16T08:05:46Z

Commit message

Test Windows using a separate workflow

We only test supported versions of GHC using the LLVM/Clang that is installed with that version of GHC.

Changes include:

Paths are computed, so that they work when GHC is installed by Chocolatey as well as GHCup. We should no longer have failures due to Chocolatey servers being down.
Pre-installed LLVM/Clang installations are removed from the PATH to ensure that we never use them.
Support for manual dispatch is added, allowing developers to specify the GHC versions to test, enable/disable cache, and set a tmate breakpoint.
Additional debugging output is added.
Steps are not flattened because all steps are defined in a single file.

TravisCardwell · 2026-02-16T08:07:50Z

What version of GHC should we test? Currently/tentatively:

PR: 9.4
MQ: 9.12
Push: 9.4 and 9.12 (Do we need this?)

Note that we cannot test using GHC 9.2 because it ships with GCC.

Note that I cannot test manual dispatch until the new workflow is in main.

TravisCardwell · 2026-02-16T08:12:31Z

A few more notes:

This workflow does not use 絵文字. I am indeed not a fan of putting them in step titles, but I did not add them because they are not even supported in my terminal. (They show up as square placeholders.)
I think that we should stop using uses: ./.github/actions/.... This is a convenience for developers because it makes it easier to keep workflows consistent when code is shared, but it has a number of drawbacks. Foremost, steps in actions are flattened, making it more difficult to read logs, which is more important than developer ergonomics IMHO. I also think that it makes understanding what is happening more difficult than a single file that clearly lists all steps and has minimal conditionals. Thoughts?

TravisCardwell · 2026-02-16T08:13:52Z

We still need to configure the required tests. Perhaps I should refactor the non-Windows tests firsts, to add support for manual dispatch if nothing else?

edsko · 2026-02-18T07:21:20Z

Note to self:

clang installed through Visual Studio ("LLVM support") will target msvc
similarly, clang installed from the official LLVM releases will also target msvc

Since ghc does not support the msvc target, neither of these is usable! If users want to use a different compiler than the one shipped with ghc, they can

install clang via MSYS [probably; we have not tested this]
build clang yourself

In this PR, we only test with bundled clang, which is 14 for ghc 9.4 .. 9.12, and clang 20 for ghc 9.14; ghc 9.2 comes with gcc, rather than clang.

For testing:

PR: 9.4 (or 9.6) and 9.14 (so that we are testing both clang versions)
MQ: The rest (though if this turns out to be too expensive, we could also decide to stick with what we already tested in the PR; I don't think it's critical).

TravisCardwell · 2026-02-18T21:33:34Z

@jorisdral

With this PR, we test the following versions of GHC on Windows, using the LLVM/Clang that is installed with each version:

PR: 9.4, 9.14
MQ: 9.6, 9.8, 9.10, 9.12

(Note: We do not test 9.2, which still uses GCC.)

Do we need to also test on push? It does not seem to be the case, because we use main as our development branch and all commits to that branch must first go through a PR and the MQ. Cache seems to work regardless. You had a comment about it, though, so I would like to confirm. If is is indeed required, it should test the union of the above sets (all versions tested in PR and MQ)?

The required checks also need to be updated. (I do not have permissions for that.)

jorisdral

@jorisdral

With this PR, we test the following versions of GHC on Windows, using the LLVM/Clang that is installed with each version:
* PR: 9.4, 9.14

* MQ: 9.6, 9.8, 9.10, 9.12
(Note: We do not test 9.2, which still uses GCC.)

I've left a review comment about which versions are tested for which triggers

Do we need to also test on push? It does not seem to be the case, because we use main as our development branch and all commits to that branch must first go through a PR and the MQ. Cache seems to work regardless. You had a comment about it, though, so I would like to confirm. If is is indeed required, it should test the union of the above sets (all versions tested in PR and MQ)?

Yes, we should run all jobs on push to main. Caches created for the default branch are available to all other branches. Caches created on a non-default branch are only available to that specific branch. So if jobs do not run on push to main, then you'd only get caching effects for re-runs on a specific branch. If main has caches then all PRs benefit from that cache. See https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache

The required checks also need to be updated. (I do not have permissions for that.)

I will do that!

jorisdral · 2026-02-19T08:33:47Z

+
+jobs:
+  setup:
+    name: "Windows: Setup"


Minor: you can choose to not prefix job/step names with Windows because the workflow already has Windows in the name

This is on purpose, because the workflow context is not always visible. The run page is used when debugging, and the workflow is only shown when you are scrolled to the very top of the page. When you are scrolled down and looking at logs, the browser title, the page title (top), and the job index (left) all just show the job name with no workflow context. When debugging, you may have a number of logs open. When you switch tabs, scrolling up to confirm the workflow can make you lose your place in the logs and is quite annoying. (Example)

Other jobs also duplicate the workflow in the name, so this is not inconsistent. IMHO, the duplication in some contexts is a small price to pay to make it easier when debugging.

jorisdral · 2026-02-19T08:40:19Z

+      - name: "[build] Build dependencies"
+        run: cabal build all --only-dependencies
+
+      - name: "[build] Build all"
+        run: cabal build all


Optimisation: save the cache right after building dependencies. None of the later steps (like cabal build all) will add anything new to the store AFAICS

Suggested change

- name: "[build] Build dependencies"

run: cabal build all --only-dependencies

- name: "[build] Build all"

run: cabal build all

- name: "[build] Build dependencies"

run: cabal build all --only-dependencies

# Only save a new cache if it does not exist yet

- name: "[setup] Save cache"

uses: actions/cache/save@v4

if: ${{ steps.restore-cache.outputs.cache-hit != 'true' && (github.event_name != 'workflow_dispatch' || inputs.enable_cache) }}

with:

path: ${{ steps.setup-haskell.outputs.cabal-store }}

key: ${{ steps.restore-cache.outputs.cache-primary-key }}

- name: "[build] Build all"

run: cabal build all

For hs-bindgen we were saving the cache at the very end because there were multiple steps that could add things to the cabal store (hs-bindgen builds, manual builds). That's not the case here, so we can save the cache much earlier. Re-runs will then always have a cache available, even if later steps failed, like building or testing

FYI: I don't think the !cancelled condition should be added here. We should only save the cache if dependencies were built successfully

This was written for consistency with the haskell workflow, and I see that has now changed. Sounds good to me! I needed to change the logic to support manual dispatch options. (When dispatching a workflow, you can disable the cache if you think it might be causing issues.)

Speaking of cache, I am not a fan of the cache-reset-number design because one needs to commit to the repo to do an admin task. I prefer to use the CLI (or web page) when a developer decides to delete cache. To avoid bloat, I generally periodically "clear cache" automatically by putting a timestamp in the key. For example, one can do so monthly (date +%Y%m), yearly (date +%Y), weekly (date +%Yw%W), etc. I am of course using cache-reset-number for consistency, but I figured I should mention that there are better ways. 🙇

Speaking of cache, I am not a fan of the cache-reset-number design because one needs to commit to the repo to do an admin task. I prefer to use the CLI (or web page) when a developer decides to delete cache. To avoid bloat, I generally periodically "clear cache" automatically by putting a timestamp in the key. For example, one can do so monthly (date +%Y%m), yearly (date +%Y), weekly (date +%Yw%W), etc. I am of course using cache-reset-number for consistency, but I figured I should mention that there are better ways. 🙇

Right, I've used that in the past as well, and I think that's totally reasonable to use an automatic reset

We only test supported versions of GHC using the LLVM/Clang that is installed with that version of GHC. Changes include: * Paths are computed, so that they work when GHC is installed by Chocolatey as well as GHCup. We should no longer have failures due to Chocolatey servers being down. * Pre-installed LLVM/Clang installations are removed from the `PATH` to ensure that we never use them. * Support for manual dispatch is added, allowing developers to specify the GHC versions to test, enable/disable cache, and set a `tmate` breakpoint. * Additional debugging output is added. * Steps are not flattened because all steps are defined in a single file. * Tests are run sequentially, to make it easier to debug failures.

TravisCardwell · 2026-02-19T22:41:02Z

The checkout step of the macOS job completely stalled. I requested to cancel the workflow, but even that did not work! Perhaps it will time out...

TravisCardwell force-pushed the tcard/windows branch from e002566 to a29837d Compare February 9, 2026 06:17

TravisCardwell mentioned this pull request Feb 9, 2026

Windows CI jobs well-typed/hs-bindgen#1256

Merged

TravisCardwell force-pushed the tcard/windows branch from a29837d to 43a0658 Compare February 9, 2026 06:23

TravisCardwell force-pushed the tcard/windows branch from 43a0658 to a0fdb1d Compare February 9, 2026 07:52

TravisCardwell force-pushed the tcard/windows branch 3 times, most recently from 8eaa39c to aeeed3f Compare February 9, 2026 21:24

TravisCardwell force-pushed the tcard/windows branch 3 times, most recently from c0a37c3 to b5ae633 Compare February 9, 2026 23:37

TravisCardwell force-pushed the tcard/windows branch from b5ae633 to e2636fb Compare February 10, 2026 00:08

TravisCardwell force-pushed the tcard/windows branch 12 times, most recently from afe2a6b to 7dd4000 Compare February 10, 2026 05:07

TravisCardwell force-pushed the tcard/windows branch 6 times, most recently from bfa2f7b to 02dccff Compare February 16, 2026 07:56

TravisCardwell marked this pull request as ready for review February 16, 2026 08:12

TravisCardwell force-pushed the tcard/windows branch 3 times, most recently from 6c6b0da to a804c81 Compare February 17, 2026 23:28

edsko mentioned this pull request Feb 18, 2026

Windows: test with ghc 9.2 (which does not have clang bundled) #30

Open

TravisCardwell force-pushed the tcard/windows branch 2 times, most recently from f8e0cd3 to 36ad2a9 Compare February 18, 2026 21:06

TravisCardwell force-pushed the tcard/windows branch from 36ad2a9 to 61cadb2 Compare February 19, 2026 00:03

TravisCardwell changed the title ~~Experiment: Windows CI workflow~~ Windows CI workflow Feb 19, 2026

TravisCardwell self-assigned this Feb 19, 2026

jorisdral reviewed Feb 19, 2026

View reviewed changes

jorisdral mentioned this pull request Feb 19, 2026

Various CI updates #32

Merged

TravisCardwell force-pushed the tcard/windows branch from 61cadb2 to 9b46bbb Compare February 19, 2026 22:00

TravisCardwell added this pull request to the merge queue Feb 19, 2026

Merged via the queue into main with commit d47f338 Feb 19, 2026
13 of 15 checks passed

Uh oh!

Conversation

TravisCardwell commented Feb 9, 2026

Uh oh!

TravisCardwell commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TravisCardwell commented Feb 9, 2026

Uh oh!

TravisCardwell commented Feb 9, 2026

Uh oh!

TravisCardwell commented Feb 9, 2026

Uh oh!

TravisCardwell commented Feb 9, 2026

Uh oh!

TravisCardwell commented Feb 10, 2026

Uh oh!

TravisCardwell commented Feb 10, 2026

Uh oh!

TravisCardwell commented Feb 16, 2026

Uh oh!

TravisCardwell commented Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TravisCardwell commented Feb 16, 2026

Uh oh!

TravisCardwell commented Feb 16, 2026

Uh oh!

TravisCardwell commented Feb 16, 2026

Uh oh!

edsko commented Feb 18, 2026

Uh oh!

TravisCardwell commented Feb 18, 2026

Uh oh!

jorisdral left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jorisdral Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

TravisCardwell Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

jorisdral Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorisdral Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

jorisdral Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TravisCardwell Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

jorisdral Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

TravisCardwell commented Feb 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

TravisCardwell commented Feb 9, 2026 •

edited

Loading

TravisCardwell commented Feb 16, 2026 •

edited

Loading

jorisdral Feb 19, 2026 •

edited

Loading

jorisdral Feb 19, 2026 •

edited

Loading