Windows DLL support

[laheller]

On Windows and Visual Studio 2022 (or newer), one wants to easily build ERFA and produce a library (erfa.dll) that can be called from any language that supports language bindings.

[laheller]

@eteq @mhvk @sergiopasra
Any thoughts?

[mhvk]

@laheller - thanks for the PR. As is, this will unfortunately not work. ERFA is essentially a copy of SOFA with a different license, and we want to keep differences between the two to an absolute minimum -- see the README for the project as a whole. This means any of the regular files cannot be changed, including the erfa.h header (or it would have to be done in the copy process described in RELEASE).

Separately, whatever is added should be covered by CI, so it should be tested that *dll are produced that pass the tests.

Finally, as I work on linux not windows, I'm a bit confused: we already build for windows using meson, why do we need a different method?

[laheller]

Finally, as I work on linux not windows, I'm a bit confused: we already build for windows using meson, why do we need a different method?

Hi @mhvk

If I understand correctly, the Windows/meson build generates a static library (erfa.lib file on Windows).
My goal is to generate a dynamic link library (or a shared library, erfa.dll on Windows) that can be used for example from C#.NET using platform invoke calls, which requires a shared library.
Goal #2 is to create a C# language wrapper around the C-library functions but in a separate repo.

I checked the things little bit and I think I can rework this PR so that the original C header files are not touched. It is possible on Windows and Visual Studio by using DEF files instead of tagging the C header functions with those ugly __declspec(dllexport) symbols.

Rework in progress...

BR,

Ladislav

[mhvk]

@laheller - again, I know little about windows, but a quick google suggests it is possible to build a .dll on windows too... https://mesonbuild.com/Reference-manual_functions.html#shared_library?

I'm not sure it isn't simply a bug that we don't do so already, since we do build a shared library on linux.

[laheller]

@laheller - again, I know little about windows, but a quick google suggests it is possible to build a .dll on windows too... https://mesonbuild.com/Reference-manual_functions.html#shared_library?

I'm not sure it isn't simply a bug that we don't do so already, since we do build a shared library on linux.

@mhvk
So if we check the output of the current GitHub CI Windows-build, it clearly produces a static library only.
I will try to figure out, how to properly configure meson/ninja to produce both static and shared library.

BR,

Ladislav

[mhvk]

@laheller - see 66aefd5 for the commit that caused only the static library to be built. Let me ping @eli-schwartz since he put it there and mentioned in the commit message that

Without declspec / .def files, this produces a DLL without symbols, and
no .lib import library. It thus cannot be linked to, particularly by the
test binaries.

I guess we need to make sure both libraries get tested somehow. (I do continue to feel out of my league here, since I don't use windows. Ping @astrofrog, who may have more experience.)

[eli-schwartz]

@laheller - see 66aefd5 for the commit that caused only the static library to be built. Let me ping @eli-schwartz since he put it there and mentioned in the commit message that [...]

I am not a Windows user either. Unfortunately I don't feel out of my league at all ;) since I maintain a cross platform build system (Meson) that has to support Windows and therefore I know way too much, all things considered, about Windows.

So: one of the original rationales for adding Meson support to erfa was in fact to support Windows (annoying to do if the build system is Autotools). Introducing Meson provided that for free, as a side effect of reducing the number of lines of code (by dropping Autotools) and making the build faster and more ergonomic.

An inherent difference of windows is as this PR correctly noted, that:

• Microsoft linkers expose zero dynamic shared object (DSO / DLL) symbols by default
• Unix linkers expose all dynamic shared object symbols by default

A library with zero symbols is useless and counterproductive, obviously. That means that on Windows, you must do extra effort to produce shared libraries; on Unix, shared libraries simply work by default, but expose "everything and the whole world" in your ABI.

ABI control is possible on all platforms.

• Windows:
  • in the headers: declspec with either dllexport or dllimport (depending on whether you're making the library or using it)
  • linker options: use a *.def mapping file
• unix:
  • in the headers: gnu attributes, specifically "visibility hidden" to hide a symbol or "visibility default" to explicitly export it; the compile option -fvisibility=hidden changes the default to be like Windows, and assumes that you then set "visibility default" on all public symbols
  • linker options: use a -Wl,--version-script to pass an LD version script mapping file

Doing explicit ABI control is good, regardless of platform, since GCC can generate more efficient code if it knows a function will not be exposed as a symbol. Visibility attributes only -- the linker version script approach occurs too late to improve codegen, all it does is hide symbols from the symbol table (that does still make the runtime loader faster).

[eli-schwartz]

I STRONGLY advise to not accept a visual studio solution checked into git. This is 1500 lines worth of mostly inscrutable xml, and meson can simply generate it for you (it is an optional alternative to ninja files).

Of course, I'm biased since I'm an upstream meson developer. ;) But as a biased person I know very well that I'm also right. :D

It's simple to wire up a def file to meson, simply pass the vs_module_defs kwarg to the library. I think you want to do that anyway, to avoid having something that works in visual studio but is broken in meson!

[eli-schwartz]

I guess we need to make sure both libraries get tested somehow. (I do continue to feel out of my league here, since I don't use windows. Ping @astrofrog, who may have more experience.)

The meson CI will already test the static library, and you can add a CI matrix to build on windows with default_library=shared and it will test the DLL version as well (because the test will try to link a test executable to the DLL and then run it). Obviously that will only work once ABI control for windows is implemented.

[laheller]

@eli-schwartz @mhvk
As Eli wrote, in case of Windows, a shared library and the meson combination probably the best would be to use a DEF file. I am workimg on it.

[mhvk]

@eli-schwartz - thanks so much! It sounds with meson a solution is quite possible.

@laheller - ping me in when it becomes worthwhile to run CI. As @eli-schwartz suggested, we'd like not to add inscrutable files (or rather, auto-generate them as needed). But I think you're already going towards that!

[laheller]

@eli-schwartz @mhvk
I did the changes, now the CI workflow creates both static & shared libraries for Windows while the latter exports all the symbols.

Now I can focus on my original goal, to create a C# .NET wrapper around the ERFA C-language functions. It will be a new repo that will reference this one. Or... what about to add add C# language support here? What do you think?

[mhvk]

Thanks! I'll let CI run, to see if tests pass.

Still some comments, though: this PR adds two files which will have to be kept up to date whenever a new release is made. Ideally, there is just one source of truth.

[mhvk]

Now I can focus on my original goal, to create a C# .NET wrapper around the ERFA C-language functions. It will be a new repo that will reference this one. Or... what about to add add C# language support here? What do you think?

My tendency would be to keep this repo as simple as it can be, and have wrappers elsewhere (like we have pyerfa and I think there are Julia wrappers as well). But I basically know nothing about C#, so if it is more like C++ where one adds a few lines to a header file, the story might be different. In any case, I would suggest to experiment outside of here first, to see what is actually needed.

[laheller]

Now I can focus on my original goal, to create a C# .NET wrapper around the ERFA C-language functions. It will be a new repo that will reference this one. Or... what about to add add C# language support here? What do you think?

My tendency would be to keep this repo as simple as it can be, and have wrappers elsewhere (like we have pyerfa and I think there are Julia wrappers as well). But I basically know nothing about C#, so if it is more like C++ where one adds a few lines to a header file, the story might be different. In any case, I would suggest to experiment outside of here first, to see what is actually needed.

@mhvk

First, I will create the C# wrapper as a separate repo, including some test code based on src/t_erfa_c.c and src/t_erfa_c_extra.c.

BTW this is how a C# wrapper looks like, in the below example it references the eraA2af( ) function:

const string LibName = "erfa.dll"; // if the shared library is available, the .NET runtime loads it automatically

/// <summary>
/// Function documentation is available at:
/// <see href="https://www.iausofa.org/s/sofa_vm_c.pdf#page=23" />
/// </summary>
[DllImport(LibName, EntryPoint = "eraA2af", CharSet = CharSet.Ansi)]
public static extern void eraA2af(int ndp, double angle, StringBuilder sign, [Out] int[] idmsf);

I think, now this PR is ready to merge.

BR,

Ladislav

[mhvk]

Yes, this is getting nice and simple, and very close to being ready to merge. A few more questions in-line (none critical).

[mhvk]

What is this version? Just of the .def file?

As I mentioned, ideally this file is generated automatically. How did you create it? Might that be transferable? (If not, we can do this as follow-up.)

[laheller]

@mhvk

This is just the DLL versioning, a kind of metadata that we can add to the final DLL during the build and it can be displayed using the dumpbin /HEADERS erfa.dll command and has nothing to do with the erfa/sofa versions.

[avalentino]

shouldn't it match the soversion then?

[laheller]

@avalentino
If the soversion is defined somewhere, then yes.

[mhvk]

It is defined in meson.build (and configure.ac, but let's not worry about that...):

erfa/meson.build

Lines 10 to 20 in 1d9738b

	# The historic versions use libtool-compatible versioning.
	# This uses some gnarly math to define ABI versions, which we replicate here.
	# The general formula is:
	# libtool: C:R:A
	# -soname: (C - A).A.R
	libtool_version = [9, 1, 8]
	soversion = '@0@.@1@.@2@'.format(
	libtool_version[0] - libtool_version[2],
	libtool_version[2],
	libtool_version[1],
	)

I wonder if this file can be generated automatically by meson (maybe with help of a script)...

[eli-schwartz]

The question is what are you going to parse and what rules to apply. :) The header file? All symbols that have the word "erfa" in them?

[mhvk]

@eli-schwartz - in our case I think ls src/*.c basically gives us what we need (strip .c, capitalize first character, and prefix with era). I think that should not be too difficult to do from meson (if just by calling out to some few-line script), but will admit my meson-fu is quite lacking (still too used to make...).

But I was asking mostly since @laheller presumably made this file one way or another, so perhaps he already used some handy one-liner. If not, I'll have a look myself, and push either to this PR or do a follow-up. If it turns out to be less easy than I think, I can always adjust the scripts that do the translation of sofa to erfa, at https://github.com/liberfa/erfa-fetch, to also create this file.

[laheller]

@mhvk
If you ask, how I made the DEF file:
I used the .NET version of the libclang library to parse the erfa.h and erfaextra.h headers to easily get the C function names (+ the whole signature needed for the C# wrapper).

[laheller]

Yes, this is getting nice and simple, and very close to being ready to merge. A few more questions in-line (none critical).

@mhvk

All resolved.

[mhvk]

The following python script reproduces windows.def exactly:

def get_names(lines):
    starts = {"void", "int", "double", "const char*"}
    return ["era" + lp[2].partition("(")[0] for l in lines
            if (lp := l.partition(" era"))[0] in starts]


with open("src/erfa.h") as fh:
    hnames = get_names(fh.readlines())

with open("src/erfaextra.h") as fh:
    xnames = get_names(fh.readlines())

with open("check.def", "wt") as fw:
    fw.write("VERSION 0.1\nEXPORTS\n")
    for l in hnames+xnames:
        fw.write(f"\t{l}\n")

[mhvk]

I'm still somewhat confused about what we need to do with VERSION, since DLL seems to do it differently from libtool. https://gnuwin32.sourceforge.net/versioning.html suggests using the first libtool number (current-age) as part of the file name.

[laheller]

@mhvk
If I understand correctly, you would like to have an autogenerated DEF file by meson + DLL version, right?

[mhvk]

@laheller - yes, I'd like to auto-generate the .def, but I think the script that I posted above will do it. For me, the only issue outstanding is what should be the version for the DLL file (see previous message).

[laheller]

@laheller - yes, I'd like to auto-generate the .def, but I think the script that I posted above will do it. For me, the only issue outstanding is what should be the version for the DLL file (see previous message).

@mhvk
Honestly, I don't know. One thing is sure: The DLL version string has only 2 components, X.Y (Major.Minor), according to the official documentation:
https://learn.microsoft.com/en-us/cpp/build/reference/version-c-cpp?view=msvc-170