StructLayouts that differ between architectures and/or operating systems

[DaZombieKiller]

Today, if you need to wrap a native API that has differing structure layouts (namely, structure packing) between platforms, you're stuck in a very messy scenario with not many good options to proceed. An example of such a scenario is Valve's Steamworks API, which has a series of callback structures with packing that varies per platform (8 on Windows, 4 everywhere else).

So, when faced with this problem, some solutions that come to mind are:

• Ship per-RID assemblies
  • Not realistically feasible outside of the runtime.
• Duplicate the structures per platform
  • This requires duplication of function signatures that use these structures by value.
  • It also requires duplication of any other structures that use these structures.
• Use interfaces
  • Results in boxing, which may or may not be acceptable depending on your scenario.
  • Boxing can be avoided by using generics, but this results in vastly more complicated bindings.

None of these are particularly appetizing, and all of them have significant drawbacks. Ideally, the runtime could provide some method to annotate structures to allow their layout to change depending on the current platform. If we ignore everything except structure packing, such a feature could look like:

[StructLayout(LayoutKind.Sequential, Pack = 4)]
[StructPackOverride(PlatformID.Win32NT, Pack = 8)]
public struct SomeCallbackStructure {}

Which would define a struct with a packing value of 8 on Windows, and 4 everywhere else.

cc @jkoritzinsky @AaronRobinsonMSFT

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[AaronRobinsonMSFT]

This seems like a great opportunity for customization in our struct marshalling source generator that @jkoritzinsky is working on. I'm hesitant to change any of our built-in marshalling support to enable this sort of functionality.

/cc @elinor-fung

[DaZombieKiller]

I'm hesitant to change any of our built-in marshalling support to enable this sort of functionality.

I should mention that for my specific use cases I am working purely with blittable structures, avoiding marshaling where I can. So ideally this would affect both the managed and unmanaged layouts.

[AaronRobinsonMSFT]

@DaZombieKiller Let me elaborate on my primary concern about built-in support. The Pack details are apart of the metadata – StructLayoutAttribute is a pseudo-attribute. This means it is sort of ugly to have something clearly declared in metadata but have another attribute that overrides that setting. It makes respecting this new attribute cumbersome since now metadata must be confirmed to be "correct" since the user could override that.

In order to improve this we would likely need some sentinel value to indicate – look somewhere else for the value. That isn't impossible but would require thought and coordinate with the Roslyn team.

[DaZombieKiller]

That makes a lot of sense, yeah. The example in the OP was more of a "this is what a potential solution to this could look like", moreso than a concrete proposal. If supporting differing layouts in the runtime itself proves to be too much of a headache, having APIs akin to StructureToPtr/PtrToStructure (which I imagine would be required by the generator if things were to go that route) would certainly still be enough to make this scenario far less of a headache to deal with.

[tannergooding]

Its likely also worth considering in what scenarios this occurs. If such APIs appear as part of particularly "hot calls", then it may be problematic to require marshalling to occur. It can also impact where changes become visible if pointers, refs, or arrays are involved.

I'd expect that work in this area might not impact interop at all (at least directly), but would by relegated to the class layout logic in the VM. Noting that touching the class layout logic is likely just as concerning overall since touching that logic has historically been semi-problematic 😄

Much of this could be simpler if it was more trivial to support per-RID assemblies in .NET (although even that can have problems due to doubling size of things generally for a 1-or-2 types that need the customization).

[jkotas]

[StructPackOverride(PlatformID.Win32NT, Pack = 8)]

Note that the conditions that express the platform differences can be quite complex. For example "Windows and not ARM" here:

runtime/src/libraries/System.Data.OleDb/src/OleDbStruct.cs

Lines 112 to 116 in dfd618d

	#if (WIN32 && !ARCH_arm)
	[StructLayoutAttribute(LayoutKind.Sequential, Pack = 2)]
	#else
	[StructLayout(LayoutKind.Sequential, Pack = 8)]
	#endif

. I think it would be hard to make the declarative scheme like StructPackOverride to be general purpose enough.

[DaZombieKiller]

Note that the conditions that express the platform differences can be quite complex.

This is a good point, and it got me thinking about potential alternatives. One that came to mind was something along the lines of:

public interface IRuntimeStructLayout
{
    static abstract int Pack { get; }
    static abstract int Size { get; }
    static abstract CharSet CharSet { get; }
}

public sealed class RuntimeStructLayoutAttribute<T> : Attribute
    where T : IRuntimeStructLayout
{
    public int Pack => T.Pack;
    public int Size => T.Size;
    public CharSet CharSet => T.CharSet;
}

Which would be used like this:

[RuntimeStructLayout<SomeCallbackLayout>]
public struct SomeCallback
{
}

sealed class SomeCallbackLayout : IRuntimeStructLayout
{
    public static int Pack => OperatingSystem.IsWindows() ? 8 : 4;
    public static int Size => 0;
    public static CharSet CharSet => CharSet.None;
}

I think such an approach would be fine if it were to be used for marshalling, but it's overall an unsatisfying solution because the requirement of code execution to get the values feels like it would be unsuitable for the managed layout (and thus wouldn't affect access through ref, pointers, etc.)

[jkotas]

This solution would be incompatible with AOT compilation. It is equivalent to emitting the type definitions at runtime using Reflection.Emit. If fact, you can implement a solution like this using Reflection.Emit today: move the types with dynamic layout into a separate assembly, do not include this assembly in the application and instead emit it using Reflection.Emit at runtime. It is not something that I would recommend.

[DaZombieKiller]

Yeah, AOT was another concern I had. I suppose it could be theoretically possible if there were heavy restrictions placed on what the IRuntimeStructLayout methods could do, but it'd be a monumental (and likely non-worthwhile) effort that would be better served investigating other options. The Reflection.Emit trick sounds clever (I assume you'd compile against a ref assembly that would look similar to what you emit at runtime?), but definitely not something I would want to pursue in a production codebase.

[danmoseley]

Ship per-RID assemblies
Not realistically feasible outside of the runtime.

Could you say more about this? Doesn't Nuget hide the complexity from the consumer here?