Provide wheels for other platforms

[eliotwrobson]

See discussion here.

[Kai-Striega]

@eliotwrobson I'm going to bring the discussion about this here, rather than spamming the other reviewers.

I'll dig into this for a bit, but I won't have the time to look into it today (I'll look at the SPEC-7 stuff), or for the next few days. But I will get to it once I have time.

Thank you! This would be greatly appreciated 🙏🏽 🙏🏽

You're welcome.

I'd be open to switching the build system as well if that is necessary,

I don't think it will be necessary, but I think it will be better. From my experience so far, Poetry doesn't work that well with Cython. (unfortunately, it's a great tool otherwise). I normally recommend spin, but that means you'd have to learn (some) meson.

I'm just currently not sure how to set up automated publishing with respect to this.

spin has a build command that builds your library for you. However, in the documentation it only lists sdist. I'll need to see how that works. It might work out of the box, otherwise, you can add a custom command to build it. I'll reach out to some people who might be able to figure it out.

[eliotwrobson]

I don't think it will be necessary, but I think it will be better. From my experience so far, Poetry doesn't work that well with Cython. (unfortunately, it's a great tool otherwise). I normally recommend spin, but that means you'd have to learn (some) meson.

No worries about learning meson, I'm totally open to switching tools. I'm going to start another project before too long that will be using Cython as well, so being able to learn / copy over infrastructure from this project would make that much easier later.

I'm also occupied with a couple of other things at the moment (including the somewhat more pressing feature releases here), so no rush 👍🏽

[Kai-Striega]

I think what you want is the cibuildwheel project. And then the standard workflow using twine.

[eliotwrobson]

@Kai-Striega thanks for the pointers so far! I've started work on this in #34, I think I have the hang of Meson (it's been pretty straightforward to learn), and I think I have the project building using hatch along with the hatch-meson plugin. It looks like the project is building correctly, but when I go to run the test cases in hatch, it looks like the compiled extensions are missing. Do you know of any projects using a similar setup, or whether I have something misconfigured in meson?

[Kai-Striega]

I don't know any project using hatch :(

I'm pretty busy this week with work, I'll try to take a more thorough look at this once I have time. Hopefully within the next fortnight.

[eliotwrobson]

@Kai-Striega no worries! I found an example project using PDM + Meson so I'm going to give that a try first 👍

[eliotwrobson]

A quick update: In the current PR #34, I've managed to get the tests and linting working with PDM, but I'm getting the following error message when I try to run cibuildwheel:

It looks like the compiled extension isn't getting installed for this test for some reason? These same tests work when I run pdm install, and it looks like the compiled file is present in the wheel files themselves. @Kai-Striega have you ever seen anything like this before?

[eli-schwartz]

It looks like the compiled extension isn't getting installed for this test for some reason? These same tests work when I run pdm install, and it looks like the compiled file is present in the wheel files themselves. @Kai-Striega have you ever seen anything like this before?

It actually shouldn't matter whether the extension is getting installed. cibuildwheel intentionally runs the tests via test-comand = "pytest {project}/tests" as documented:

Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. To ensure the wheel is imported by your tests (instead of your source copy), tests are not run from your project directory. Use the placeholders {project} and {package} when specifying paths in your project. If this variable is not set, your wheel will not be installed after building.

So, /project/cynetdiff/utils.py shouldn't be on python's sys.path as from cynetdiff.utils import ...... at all. The fact that it is, implies something else went wrong.

If it was just the wheel not being installed, I would assume that it couldn't import cynetdiff at all -- not that it would import an incorrect version.

[eliotwrobson]

So, /project/cynetdiff/utils.py shouldn't be on python's sys.path as from cynetdiff.utils import ...... at all. The fact that it is, implies something else went wrong.

If it was just the wheel not being installed, I would assume that it couldn't import cynetdiff at all -- not that it would import an incorrect version.

@eli-schwartz thanks for your comment! how would you expect to be able to import the utils file given the current configuration? I should be able to play around with this some more in a few days and try debugging some more. Do you know of any other projects I could use as a reference?

[eliotwrobson]

Resolved with #36