How to Use Cython Generated Header Files in Other Libraries

[WillAyd]

In Cython, usage of the public and api keywords can help auto-generate header files which help include the Cython module in C/C++ modules. This feature is documented here:

https://docs.cython.org/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c

With Meson, those auto-generated headers are appearing in the build directory. However, it's unclear to me how you should be directing other targets to use those auto-generated headers.

For a practical example, I've been working with the Arrow project to build their Python library in Meson. They have a file lib.pyx which generates lib.pyx.h and lib.pyx_api.h files in the build directory when invoked via:

lib_lib = py.extension_module(
    'lib',
    sources: ['lib.pyx'] + pyarrow_srcs,
    include_directories: ['src'],
    cython_args: cython_args,
    dependencies: [arrow_dep, numpy_dep],
    override_options: ['cython_language=cpp'],
    subdir: 'pyarrow',
    install: true,
)`

How can I let then subsequent targets know where in the build directory to find those headers?

[eli-schwartz]

Which directory do they appear in?

Can you check out build_tgt.private_dir_include()?

[eli-schwartz]

Good question -- does cython automatically put it in {modulename}/ for you?

[WillAyd]

Yea it appears to do that. I thought originally that would be tied to the subdir argument, but even removing that will place the files in the {modulename} directory.

[eli-schwartz]

Then possibly cython expects you to include the modulename in your #include anyway. :D

[WillAyd]

I did just come across another limitation here. On first invocation it appears there may be a race condition between the time when cython actually generates the header files in build_tgt.private_dir_include() and when subsequent targets may try to compile against that.

Is that specific to this case or is there something more general I need to do to ensure the build sequencing is correct?

[eli-schwartz]

Normally you'd want to add an explicit dependency for other targets on the generated headers. In this particular case, meson is helpful and handles creating cython sources automatically for you, but this doesn't really handle the generated headers case.

Maybe it makes sense for this specific case to instead use a custom_target() to run cython so that you can use its outputs as order dependencies for other targets that use the header.