New Source Generator based approach to getting all the Win32 methods and types you need

[AArnott]

As announced in Making Win32 APIs More Accessible to More Languages, the Microsoft Windows SDK team now delivers a machine-readable metadata file that describes the entire Win32 SDK. This allows us to auto-generate C# p/invoke methods and supporting types rather than manually write them.

But even more interesting to users of this project, you no longer have to redistribute a bunch of PInvoke assemblies with your application. This deprecated model is stuck between not having enough of the Win32 SDK represented and having more than you app needs and bloating your shipping application. Instead, the new approach generates exactly the p/invoke methods and supporting types your app or library needs as source directly into your assembly. No bloat, and no extra runtime dependencies.

Check out the C# projection source generator that is based on the Windows SDK team's metadata over at microsoft/CsWin32.

So what happens to this project? Is there still interest in consuming pre-built assemblies via NuGet given this new alternative? We might continue to support and grow this project. We might augment or even replace its current hand-written code with the projection from CsWin32. What do you think?

Leave your comments below. Please look for thoughts you agree with and use the GitHub reaction buttons to vote up that comment rather than adding a new one, where possible. Feel free to vote down suggestions you dislike. We are entertaining all respectful suggestions here and are very interested in your feedback!

[AArnott]

Possible reason to maintain this repo: support .NET languages other than C# (since as of today, C# is the only language that supports source generation.

(vote this issue up only if it applies to you)

[AArnott]

Possible reason to maintain this repo: Avoid enabling unsafe code blocks in your own project.

(vote this issue up only if you are unwilling or unable to enable unsafe code in your C# projects)
See also microsoft/CsWin32#26 and consider voting that up if it would be meaningful to you.

[mythz]

Personally I'd still prefer consuming PInvoke nuget packages which I consider the official typed .NET API for Win32 APIs that works like a normal library that others packages can depend on to create & share higher-level functionality.

I do think the 2 initiatives should be unified so that it generates a source compatible API that projects can choose to easily move between source generators & pinvoke packages (if it's not already).

[ffMathy]

👍 I also think this project should use source generators to generate all the packages.

[AArnott]

It would certainly make maintaining this repo easier if we wholesale switched to the CsWin32 projection to build the nuget packages. There are two big caveats to this however, just to be transparent:

1. There would be an initial binary and source breaking change to the APIs offered in the nuget packages.
2. There would be an ongoing series of smaller breaking changes between releases because the metadata takes breaking changes to improve the API.

The second point deserves more explanation, so here it goes: there are still bugs in the metadata. There are missing APIs. And (more than anything) there are usability fixes to make such as moving sets of constants into enums. All of these changes potentially create breaking changes. When you are consuming a source generator directly in your project, life is good because you can directly manage when to update CsWin32 and thus when to deal with the breaking changes (if they impact you at all). But when you are consuming a .NET assembly, which generally speaking has to be shared across an entire app, breaking changes are much more difficult to deal with. Consider an app that consumes 3 libraries, each of which compiled against a different version of the PInvoke nuget packages. No matter which version the app ships, 2 of these libraries may fail at runtime because the runtime API doesn't match the compile-time API.

We could mitigate this somewhat by not jumping "all in" to 100% API coverage in this project using the projection. Instead, we could still add individual APIs at a time and check them for likelihood of future breaking changes and only add them to the package if they look stable. For example, we would reject additions from the projection that use constants that are likely to move into enums later.

[AArnott]

For you non-C# users, what would you say to the workaround of just defining a C# project in your solution for purposes of consuming CsWin32, and referencing that project from your F# (or other) language project?

[saul]

Another benefit to maintaining this repo as a fully precompiled CsWin32 is for users who do not have internet access in their dev environment but still want docs. From my understanding of the CsWin32 readme, you need internet access for it to be able to write XML docs:

Generates xml documentation based on and links back to docs.microsoft.com

[AArnott]

@saul no, the source generator contains a full copy of the docs in itself. No online access required.

[AraHaan]

I would love for it to be source generated as well (possibly also include MiniDumpWriteDump inside of dbghelp as well too as many people might want to call that api function as well). Currently Dbghelp package lacks the means of creating any sort of crash dumps manually in the code.

[AArnott]

This is the plan. It's happening in the CsWin32Projection branch.

[lostmsu]

@AArnott just an opinion, call it an educated guess if you will.

TL;DR; I am concerned about who will maintain Windows.Win32.winmd and CsWin32.

I wouldn't want this project to switch to CsWin32 until they stabilize it and maybe even receive the first patch, because I feel uncertain about CsWin32 future. This doubt is based on the fact, that .NET team which previously maintained WinRT/.NET (which CsWin32 is the replacement for) dropped it. While the real reason for that is unknown to public, I think it was because .NET team felt that supporting WinRT is too much of maintenance burden. If that is right, there's a risk that this project will have to maintain CsWin32.

Again, just guessing here, but I expect CsWin32 to be now maintained by the WinRT team. And I don't have much confidence in it, because WinRT so far looks like a major fiasco.

Now this all might be nulled if CsWin32 will not try to build an object-oriented COM-based API like WinRT did, and so will be much easier to maintain.

P.S. Arguably, this project itself already is a better Windows.Win32.winmd than the "official" thing.