Skip to content

Conversation

@joerick
Copy link
Contributor

@joerick joerick commented Jan 25, 2026

Xref #2663.

Trying to make a test that recreates the issue in #2663.

The integration test for meson probably won't recreate it, but the bin/run_example_ci_configs.py should be able to.

@joerick joerick marked this pull request as draft January 25, 2026 22:56
@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026


Examples test run

Branch: example-config-test---meson-test-2026-01-25T23-02-46

Service Config Status
github examples/github-minimal.yml Build

Generated by bin/run_example_ci_configs.py --build-backend meson examples/github-minimal.yml


@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026

Well, the example config test did fail on windows. I'm a little disheartened that the error message doesn't match the one we saw in #2663.

  The Meson build system
  Version: 1.10.1
  Source dir: D:\a\cibuildwheel\cibuildwheel
  Build dir: D:\a\cibuildwheel\cibuildwheel\.mesonpy-veirak6b
  Build type: native build
  Project name: spam
  Project version: 0.1.0
  C compiler for the host machine: C:\Strawberry\c\bin\ccache.EXE cc (gcc 15.2.0 "cc (x86_64-posix-seh-rev0, Built by MinGW-Builds project) 15.2.0")
  C linker for the host machine: cc ld.bfd 2.45
  Host machine cpu family: x86_64
  Host machine cpu: x86_64
  Program python found: YES (C:\Users\runneradmin\AppData\Local\Temp\build-env-88vgdqqj\Scripts\python.exe)
  Need python for x86_64, but found x86
  Run-time dependency python found: NO (tried pkgconfig and sysconfig)
  
  ..\meson.build:9:3: ERROR: Python dependency not found

@joerick
Copy link
Contributor Author

joerick commented Jan 25, 2026

Aha! But it is the same error as we see in #2701 ! So I think perhaps this is the same issue appearing in different ways.

@joerick
Copy link
Contributor Author

joerick commented Jan 26, 2026

Hm... well the integration test (which is unaffected by the action.yml changes) is failing with the same error-

Command line: `cc -IC:\\Users\\runneradmin\\AppData\\Local\\pypa\\cibuildwheel\\Cache\\nuget-cpython\\pythonx86.3.12.10\\tools\\Include C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-1\\popen-gw1\\test_meson_python_basic_build_0\\project\\.mesonpy-4w1qbvcz\\meson-private\\tmpstr2bsu7\\testfile.c -E -P -D_FILE_OFFSET_BITS=64 -P -O0` -> 0
Run-time dependency python found: NO (tried pkgconfig and sysconfig)

..\\meson.build:9:3: ERROR: Python dependency not found

Perhaps I'm missing something in my meson setup?

Test Meson sources here.

@joerick
Copy link
Contributor Author

joerick commented Jan 26, 2026

Ah, this error is in the win32 build. We have this warning earlier in the file-

Need python for x86_64, but found x86

So we've given meson a Python.exe that's x86, but it thinks it wants the x86_64 for some reason. I'd like to know how meson-python has decided it's targeting x86_64.

Edit: It's chosen x86_64 because the compiler it found on PATH is defaulting to that.

@joerick joerick changed the title Add an integration test with the meson backend Add an integration test and debug issues with the meson backend Jan 27, 2026
@joerick
Copy link
Contributor Author

joerick commented Jan 28, 2026

Here's what I've discovered.

By default, meson searches PATH for the first available cc compiler. On Github, even when running in pwsh, that compiler is MinGW.

Most projects out there have found ways to work around this. I've seen two ways-

  • ilammy/msvc-dev-cmd, example which pushes the VS compilers to the top of the PATH.
  • another approach, used by Matplotlib, is to pass --vsenv to setup-args using the config-settings option.

The other wrinkle is win32 builds. As far as I can tell, this isn't too easy with meson-python. Many of the scientific stack projects have already dropped win32. Most projects that support it seem to use separate builds and use ilammy/msvc-dev-cmd to choose a 32-bit toolchain before starting cibuildwheel. Curious if @rgommers has a take here?

Also I wonder if build-details.json PEP 739 could help here.

Anyway, for now, I've disabled win32 builds in this test.

I'd love to improve this interplay between cibuildwheel and meson-python. One idea I've had would be for cibuildwheel to call 'vsdevcmd.bat', like how we already do it for GraalPy. I suspect we might have to make that configurable, though. Some users might already have explicitly setup compilers on PATH.

This is so it can be tested with bin/run_example_ci_configs.py
@joerick
Copy link
Contributor Author

joerick commented Jan 28, 2026


Examples test run

Branch: example-config-test---meson-test-2026-01-28T09-28-54

Service Config Status
github examples/github-minimal.yml Build

Generated by bin/run_example_ci_configs.py --build-backend meson examples/github-minimal.yml


@rgommers
Copy link

Thanks for the ping @joerick. Yes I can see a few possible solutions. I don't think build-details.json is relevant here (meson-python can easily know the bitness of the interpreter either way). And gh-2673 should probably be merged independent of whatever solution is chosen, since it fixes regressions and is a much cleaner fix?

By default, meson searches PATH for the first available cc compiler.

That's one of the way. The others ways are:

  1. Native and cross files containing c = /path/to/c/compiler
  2. Checking for the relevant environment variable(s): CC, CXX
  3. Known compiler names on PATH
  4. If nothing was found for (3), try auto-activating MSVC on Windows (x64 on 64-bit Windows)

Note that if --vsenv is passed, that selects MSVC before checking on PATH - but always 64-bit MSVC on 64-bit Windows.

I'll also note that CMake is quite similar to Meson with these options, and this scikit-build-core issue described roughly the same problem: scikit-build/scikit-build-core#668.

I suspect we might have to make that configurable, though. Some users might already have explicitly setup compilers on PATH.

Agreed, it's quite hard to change defaults without breaking some build setup or use case for other users. You can't really go check all the ways existing compilers may already be configured. So opt-in is a safer. Or, if the backwards compat impact seems acceptable, then there should be an easy opt-out at the very least.

Meson is a generic build system, it's not going to change its compiler selection logic here (it's been considered and rejected, for good reasons). That means that if we want a more polished user experience by special-casing 32-bit Python on 64-bit Windows, we can either do it in cibuildwheel, or in meson-python. And as opt-out or opt-in.

There is one relevant Meson issue which would make it easier for the user to opt in to using 32-bit MSVC: mesonbuild/meson#11435 (this comment in particular).

(Second post with solution options coming)

@rgommers
Copy link

rgommers commented Jan 28, 2026

Actually, on second thought, the list of solution options is shorter than I thought.

I think the main one is to do both of these:

  1. Extending Meson's built-in functionality so one can easily do --vsenv=x86 or --vsenv=arm64, as discussed in meson#11435
  2. meson-python to check if the user added --vsenv, and if so then checking the target interpreter - and if it's 32-bit Python on Windows, then use --vsenv=x86. The same might apply to compiling for arm64 on an x64 machine with emulation (not 100% sure).

That makes it very easy for package authors to opt in to the automatic selection by using --vsenv in their pyproject.toml.

Cc @eli-schwartz and @dnicolodi for thoughts on this.

I'd love to improve this interplay between cibuildwheel and meson-python. One idea I've had would be for cibuildwheel to call 'vsdevcmd.bat',

I think that this is also reasonable to do, especially if it's easy and it's already being done for GraalPy. I'm not 100% sure that it won't be overridden anyway by --vsenv though.

@eli-schwartz
Copy link

Please note that the --vsenv command line option to meson instructs meson to pass force=True to the MSVC activation logic.

The MSVC activation logic is unconditionally run for all builds on all platforms. It:

  • returns False immediately if one is not on a Windows operating system
  • returns False immediately if cl.exe is on PATH and MSVC is already set up
  • checks if force=True and if so, skips checking on PATH for a GCC or clang or clang-cl compiler, otherwise it will early return if it finds one
  • tries to activate MSVC

Thus, passing --vsenv as a command line parameter to meson does not cause meson to detect MSVC, it just causes meson to prefer to detect MSVC even if it is "not on PATH && another compiler is on PATH".

@eli-schwartz
Copy link

Meson is a generic build system, it's not going to change its compiler selection logic here (it's been considered and rejected, for good reasons). That means that if we want a more polished user experience by special-casing 32-bit Python on 64-bit Windows, we can either do it in cibuildwheel, or in meson-python. And as opt-out or opt-in.

Yup -- note probably the biggest practical (rather than philosophical) issue here is that compiler detection has to happen during project() which occurs before import('python').find_installation(), after which it's too late.

I think the main one is to do both of these:

  1. Extending Meson's built-in functionality so one can easily do --vsenv=x86 or --vsenv=arm64, as discussed in meson#11435

This seems reasonable to me.

  1. meson-python to check if the user added --vsenv, and if so then checking the target interpreter - and if it's 32-bit Python on Windows, then use --vsenv=x86. The same might apply to compiling for arm64 on an x64 machine with emulation (not 100% sure).

This sounds like it might require parsing the meson command line and also project(default_options: [...]) and I'm not sure meson-python wants to go there. Although if it's just parsing the default setup arguments from pyproject.toml's tool.meson-python table then sure, that should be easy to do.

I think that this is also reasonable to do, especially if it's easy and it's already being done for GraalPy. I'm not 100% sure that it won't be overridden anyway by --vsenv though.

As per my previous comment, this will be fine.

@rgommers
Copy link

Although if it's just parsing the default setup arguments from pyproject.toml's tool.meson-python table then sure, that should be easy to do.

Yeah, that's what I was thinking, and is as far as I would go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants