Skip to content

Meson build, replacing setuptools#109

Merged
marktsuchida merged 3 commits into
mainfrom
meson
Jul 17, 2025
Merged

Meson build, replacing setuptools#109
marktsuchida merged 3 commits into
mainfrom
meson

Conversation

@marktsuchida

@marktsuchida marktsuchida commented Jan 19, 2024

Copy link
Copy Markdown
Member

Limitation: The sdist (source distribution for PyPI) now contains the whole of mmCoreAndDevices and more (33 MiB). While technically functional, we should fix this before merging this change.

Update mmCoreAndDevices to latest (which has meson.build for MMCore and MMDevice).

Add Meson build file.

Use meson-python so that python -m build just works via pyproject.toml.

Move the single source of truth for the version number from _version.py (now generated) to meson.build.

The Meson build has several advantages:

  • Build details of MMDevice and MMCore come from their own build files, rather than being duplicated here in setup.py
  • Editable installs truly work (even if C++ files are edited) (Caveat: beware of importing pymmcore from the source root)
  • Since we are using a true C++ build system, controlling build options is much easier and cleaner than it was with setuptools

The main disadvantage is that MANIFEST.in can no longer be used to control what gets included in the sdist (meson-python uses meson dist to produce the sdist, which includes all version-controlled files). However, once we are ready to use MMDevice and MMCore from independent repositories, this will no longer be an issue (and ends up being simpler than the error-prone MANIFEST.in).


  • Build works on macOS
  • Build works on Linux
    • Need dependency('threads')
    • Upstream (MMCore) needs dependency('threads')
  • Build works on Windows
    • Prevent Meson from looking for static libpython
    • Get CI to use MSVC, not MinGW, for build
  • Disable MMDevice/MMCore unit tests (by default) when building pymmcore (to avoid including Catch2)
    • Upstream (MMDevice, MMCore) needs to have option to disable tests
  • sdist contains only the needed files

@marktsuchida

Copy link
Copy Markdown
Member Author

Rebased and squashed.

Regarding the remaining issue (cannot produce correct sdist that doesn't include the massive mmCoreAndDevices), there are 3 possible solutions I can think of:

  • Wait until we have MMDevice and MMCore in separate Git repositories (and it should just work)
  • Come up with some ad-hoc way to produce the sdist outside of the regular build (yuck, and also error-prone if not using automatic check on MANIFEST.in)
  • Come up with our own PEP-517 build backend that forwards the sdist build to setuptools and wheel builds to meson-python (more work than I'd like to do)

Given that this Meson-based build does not add any user-facing benefits, I'm inclined to take the first approach and keep this draft PR open and maintained until we're ready. It has already served to iron out at least some of the issues we'll likely encounter when moving MMCoreJ to Meson.

@marktsuchida

marktsuchida commented Oct 8, 2024

Copy link
Copy Markdown
Member Author

Rebased to current main.

  • Remove the temporary enablement of wheel building in PR

@marktsuchida

marktsuchida commented Jul 16, 2025

Copy link
Copy Markdown
Member Author

Rebased, and added commit to eliminate the mmCoreAndDevices submodule and instead use the mmcore mirror repo (micro-manager/mmCoreAndDevices#687) via Meson wrap.

@marktsuchida marktsuchida force-pushed the meson branch 2 times, most recently from 77cd1e7 to a586983 Compare July 16, 2025 17:05
@marktsuchida

Copy link
Copy Markdown
Member Author

Removed the commit to temporarily build wheels on PR; see this build for wheels.

@marktsuchida marktsuchida marked this pull request as ready for review July 16, 2025 17:07
Add Meson build file.

Use meson-python so that `python -m build` just works via
pyproject.toml.

Move the single source of truth for the version number from _version.py
(now generated) to meson.build.

The Meson build has several advantages:
- Build details of MMDevice and MMCore come from their own build files,
  rather than being duplicated here in setup.py
- Editable installs truly work (even if C++ files are edited) (Caveat:
  beware of importing pymmcore from the source root) (Another caveat:
  with uv, caching may cause tricky issues)
- Since we are using a true C++ build system, controlling build options
  is much easier and cleaner than it was with setuptools

The main disadvantage is that MANIFEST.in can no longer be used to
control what gets included in the sdist (meson-python uses `meson dist`
to produce the sdist, which includes all version-controlled files). The
sdist (source distribution for PyPI) now contains the whole of
mmCoreAndDevices and more (33 MiB). While technically functional, we
should fix this before merging this change.
This fixes the sdist; it now contains just the needed files.

Note that an mmdevice.wrap is explicitly included, in order to override
the one provided by mmcore, so that we pin the version.
@marktsuchida

Copy link
Copy Markdown
Member Author

Last force-push touches only maintainer-notes.md.

@marktsuchida marktsuchida merged commit dc38fd9 into main Jul 17, 2025
17 checks passed
@marktsuchida marktsuchida deleted the meson branch July 17, 2025 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant