[API Proposal]: Type Mapping API

[AaronRobinsonMSFT]

Background and motivation

The TypeMap is a solution for large interop based frameworks (for example, Java, Swift, WinRT, etc). The issue is attempting to map a type from another platform to the .NET platform and vice versa. The classic COM scenario uses the Guid type as the common key. Note that all .NET types have an associated GUID and therefore one can easily map from Type to Guid with minimal effort. There are two interop scenarios to consider the reverse P/Invoke (unmanaged to managed) or P/Invoke (managed to unmanaged).

For the reverse P/Invoke some type from another environment is entering the managed environment, what type do we map this instance to?
Examples: Guid -> Type (COM) or string -> Type (WinRT)

The P/Invoke means a Type is leaving managed code and being projected into an unmanaged environment, what/how do we project?
Examples: Type -> CCW (built-in COM) or Type -> Type (Where the second Type is the ABI projection, for example WinRT).

Both scenarios above can be relatively simple to solve in a JIT-supported .NET environment. However, for AOT scenarios, where size is a consideration, the above mapping must be trimmable and the simple type mapping solutions using tools like Type.GetType() are not viable.

This API is designed to both support the Type Mapping needed for interop as well as be trimmable for AOT scenarios.

See #110691 for complete details.

Comment describing the workflow this supports - #110691 (comment)

API Proposal

namespace System.Runtime.InteropServices;

/// <summary>
/// Type mapping between a string and a type.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAttribute<TTypeUniverse> : Attribute
{
    /// <summary>
    /// Create a mapping between a value and a <see cref="System.Type"/>.
    /// </summary>
    /// <param name="value">String representation of key</param>
    /// <param name="target">Type value</param>
    /// <remarks>
    /// This mapping is unconditionally inserted into the type map.
    /// </remarks>
    public TypeMapAttribute(string value, Type target)
    { }

    /// <summary>
    /// Create a mapping between a value and a <see cref="System.Type"/>.
    /// </summary>
    /// <param name="value">String representation of key</param>
    /// <param name="target">Type value</param>
    /// <param name="trimTarget">Type used by Trimmer to determine type map inclusion.</param>
    /// <remarks>
    /// This mapping is only included in the type map if the Trimmer observes a type check
    /// using the <see cref="System.Type"/> represented by <paramref name="trimTarget"/>.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public TypeMapAttribute(string value, Type target, Type trimTarget)
    { }
}

/// <summary>
/// Declare an assembly that should be inspected during type map building.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAssemblyTargetAttribute<TTypeUniverse> : Attribute
{
    /// <summary>
    /// Provide the assembly to look for type mapping attributes.
    /// </summary>
    /// <param name="assemblyName">Assembly to reference</param>
    public TypeMapAssemblyTargetAttribute(string assemblyName)
    { }
}

/// <summary>
/// Create a type association between a type and its proxy.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = false)]
public sealed class TypeMapAssociationAttribute<TTypeUniverse> : Attribute
{
    /// <summary>
    /// Create an association between two types in the type map.
    /// </summary>
    /// <param name="source">Target type.</param>
    /// <param name="proxy">Type to associated with <paramref name="source"/>.</param>
    /// <remarks>
    /// This mapping will only exist in the type map if the Trimmer observes
    /// an allocation using the <see cref="System.Type"/> represented by <paramref name="source"/>.
    /// </remarks>
    public TypeMapAssociationAttribute(Type source, Type proxy)
    { }
}

/// <summary>
/// Entry type for interop type mapping logic.
/// </summary>
public static class TypeMapping
{
    /// <summary>
    /// Returns the External type type map generated for the current application.
    /// </summary>
    /// <typeparam name="TTypeUniverse">Type map universe</typeparam>
    /// <returns>Requested type map</returns>
    /// <remarks>
    /// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public static IReadOnlyDictionary<string, Type> GetOrCreateExternalTypeMapping<TTypeUniverse>();

    /// <summary>
    /// Returns the associated type type map generated for the current application.
    /// </summary>
    /// <typeparam name="TTypeUniverse">Type map universe</typeparam>
    /// <returns>Requested type map</returns>
    /// <remarks>
    /// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
    /// </remarks>
    [RequiresUnreferencedCode("Interop types may be removed by trimming")]
    public static IReadOnlyDictionary<Type, Type> GetOrCreateProxyTypeMapping<TTypeUniverse>();
}

API Usage

Provided by .NET for Android runtime

// Type used to express specific type map universe.
public sealed class JavaTypeMapUniverse { }

// Java capability to enable type instance creation.
public interface IJavaCreateInstance
{
    object Create(ref JniObjectReference reference, JniObjectReferenceOptions options);
}

// Java capability implementation for object creation.
public class JavaObjectTypeMapAttribute<[DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] T> : Attribute, IJavaCreateInstance
    where T : JavaObject
{
    public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
    {
        // Create instance using T.
    }
}

// Java capability implementation for string creation.
public class JavaStringTypeAttribute : Attribute, IJavaCreateInstance
{
    public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
    {
        IntPtr ptr = /* Extract the underlying string pointer from reference. */
        string str = Marshal.PtrToStringUni(ptr);
        return str;
    }
}

// -- Alternative attribute design
// This would expose the direct as a Type instead of as a generic parameter. It would reduce
// type loading costs and could be the single attribute for all "capabilities".
public class JavaObjectTypeMapAttribute : Attribute
{
    public JavaObjectTypeMapAttribute([DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type type)
    { }

    public [DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type TargetType { get; }
}

.NET for Android projection library

// Dependent association between typeof(string) => typeof(StringProxy).
[assembly: TypeMapAssociation<JavaTypeMapUniverse>(typeof(string), typeof(StringProxy))]

// Map string name of Java type to typeof(StringProxy), using typeof(string) to indicate
// if the entry should be included or not.
[assembly: TypeMap<JavaTypeMapUniverse>("java/lang/String", typeof(StringProxy), typeof(string))]

[JavaStringType]
public class StringProxy
{ }

User application

// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeMapUniverse>("Application.SideCar.dll")]

// User defined type
[JavaObjectTypeMap<MyJavaClass>]
public class MyJavaClass : JavaObject
{ }

var inst = new ClassFromNuGet();

Generated Application.SideCar.dll

A sidecar assembly could always be generated to handle the following:

• Specify assembly type map targets (for example, .NET for Android runtime)
• Specify type mappings for types defined in the application.
• Generate proxies and mappings for types in Nugets to support older TFMs.

// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeMapUniverse>("Android.Runtime.dll")]

[assembly: TypeMap<JavaTypeMapUniverse>("app/MyJavaClass", typeof(MyJavaClass), typeof(MyJavaClass))]

[assembly: TypeMap<JavaTypeMapUniverse>("lib/classFromNuget", typeof(ClassFromNuGetProxy), typeof(ClassFromNuGet))]

[JavaObjectTypeMap<ClassFromNuGet>]
internal class ClassFromNuGetProxy
{ }

Interop runtime (for example, .NET for Android) usage example.

// Stored by the interop runtime in some global static
static IReadOnlyDictionary<string, Type> s_JavaTypeMap = TypeMapping.GetOrCreateProxyTypeMapping<JavaTypeUniverse>();

...

string javaTypeName = /* Get Java type name from jniHandle */
s_JavaTypeMap.TryGetValue(javaTypeName, out Type? projection);
Debug.Assert(projection != null); // Assuming map contains type.

var creator = (IJavaCreateInstance)projection.GetCustomAttribute(typeof(JavaObjectTypeMapAttribute<>));
var objectRef = new JniObjectReference(jniHandle, JniObjectReferenceType.Local);
JavaObject mcw = (JavaObject)creator.Create(ref objectRef, JniObjectReferenceOptions.None);

Alternative Designs

API design alternatives:

• The APIs on TypeMapping could instead be getter methods instead of exposing the IReadOnlyDictionary<,>.
• The APIs on TypeMapping could avoid the Try semantics and always return the IReadOnlyDictionary<,> since there is expected to be a dynamic solution that will create the IReadOnlyDictionary<,> at run-time.
• Generics could be removed from several APIs to avoid the additional loading of types for each universe combination.

API alternative
Continue with the bespoke solutions that currently exist in CsWinRT, Android (Java), Swift, and future interop platform targets.

Risks

This has a minimal risk to the current ecosystem as it is new.

The larger risk is creating something that doesn't address all issues for current or future interop platform targets. At present, there is work being done CsWinRT to confirm the mechanics. The Android (Java) target is actively being explored with no obvious concerns. The Trimmer team has been consulted extensively about this API and signed off from the trimmable perspective.

Related to missing a functional need, there is a potential to be less performant than the current bespoke solutions being employed. This performance gap could impact adoption.

Finally, since the trimmable semantics are very strict in this model, it is possible incorrectness in existing TypeMap implementations may be uncovered resulting in new issues for adopting interop frameworks.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

[AaronRobinsonMSFT]

@agocke @sbomer @rolfbjarne @jonpryor @jkoritzinsky @MichalStrehovsky @Sergio0694 @jkotas @simonrozsival @ivanpovazan

I'd like to get this API approved ASAP. Please prioritize questions/comments/concerns.

[Sergio0694]

From the CsWinRT side, we've been busy prototyping CsWinRT 3.0 on top of these APIs (see fhl/cswinrt-3.0), and at least from a usability perspective, we're actually quite happy with how well the design integrates with things on our end. Obligatory: we couldn't actually test this (for obvious reasons), but the API shape seems to cover all scenarios we've looked at so far 😄

A couple notes while at it:

• I have a slight preference for GetExternalTypeMapping returning the type map directly, over TryGetExternalTypeMapping, if we decide to just always let the trimmer build a map or throw an exception if it's missing. Mostly because it makes it a little bit more convenient to use it in static field initializers (since my understanding is, you should cache the type map returned from those methods?). Otherwise yeah we can also live with the TryGet___ pattern if there's a strong preference.
• I was talking with @MichalStrehovsky and he mentioned the linker might have to be a bit conservative with respect to trimming entries in the external type map in some cases, causing it to keep more entries for some types even if there's no actual type check for them anywhere. I suppose this is more related to docs, but it'd be nice to have a detailed explanation somewhere of what patterns one should try to prefer or avoid to make the trimmer be more effective here, what could cause it to root unwanted things, etc.
• Similar to the previous point, I'd like some more clarify on how generics come into play here. In CsWinRT, we have a bunch of helpers where it'd be convenient for us to have a generic method which also does some casting (eg. think return (T)foo). I understand that the linker wouldn't "see" those casts with respect to the external type table, which is great, exactly what we want (we should document this too). But are there other considerations with respect to casts in generics? Would they cause the trimmer to keep more stuff in either type map, and if so, in what scenarios exactly?

Really looking forward to this API, thank you again @AaronRobinsonMSFT for driving this!

[MichalStrehovsky]

I have a slight preference for GetExternalTypeMapping returning the type map directly, over TryGetExternalTypeMapping, if we decide to just always let the trimmer build a map or throw an exception if it's missing. Mostly because it makes it a little bit more convenient to use it in static field initializers (since my understanding is, you should cache the type map returned from those methods?). Otherwise yeah we can also live with the TryGet___ pattern if there's a strong preference.

I was thinking about that one too. Unless there's a special need for the API to return a dictionary, it might be better to do:

    public static bool TryGetExternalTypeMapping<TTypeUniverse>(string id, [NotNullWhen(returnValue: true)] out Type type)
        where TTypeUniverse : ITypeMapUniverse;

    public static bool TryGetTypeProxyMapping<TTypeUniverse>(Type key, [NotNullWhen(returnValue: true)] out Type value)
        where TTypeUniverse : ITypeMapUniverse;

Returning an IReadOnlyDictionary makes it possible for someone to write code that iterates it or counts things in it and unless that's needed for something, it's just a way to write code that may not behave the same before/after trimming.

I was talking with @MichalStrehovsky and he mentioned the linker might have to be a bit conservative with respect to trimming entries in the external type map in some cases, causing it to keep more entries for some types even if there's no actual type check for them anywhere. I suppose this is more related to docs, but it'd be nice to have a detailed explanation somewhere of what patterns one should try to prefer or avoid to make the trimmer be more effective here, what could cause it to root unwanted things, etc.

I don't know if this is something we could contractually guarantee. We'd consider things casted to something whenever it's possible in some way to get there. For example #110691 (comment), or IsAssignableFrom APIs, etc.

Similar to the previous point, I'd like some more clarify on how generics come into play here. In CsWinRT, we have a bunch of helpers where it'd be convenient for us to have a generic method which also does some casting (eg. think return (T)foo). I understand that the linker wouldn't "see" those casts with respect to the external type table, which is great, exactly what we want (we should document this too). But are there other considerations with respect to casts in generics? Would they cause the trimmer to keep more stuff in either type map, and if so, in what scenarios exactly?

It's my understanding that this is for casts and cast-like behaviors in user code, not just within the interop layer.

Casts in generic code are still casts. If we were to treat it as anything else, it would make things difficult to reason about. Trimming will need to be more conservative to account for generics. It already does that to keep casting in general working. I think the general rule we'll need to end up with is "if you can obtain a System.Type of the type at runtime, it should also be considered used in a cast", modulo maybe things like System.Type used in a ==/!= comparison. Whenever we optimize away a "cast", it's a potential after-trimming behavioral difference, so we'd want as few of them as possible, ideally none.

The proposal LGTM otherwise!

[rolfbjarne]

I think I see a potential problem, where the assembly attributes can't see nested private types for any typeof expressions:

// this won't compile
[assembly: TypeMap<ObjectiveCTypeUniverse> ("MyInnerClass", typeof (MyClass.MyInnerClass))]

class MyClass {
    class MyInnerClass : NSObject {}
}

and this won't work either:

class MyClass {
	// this won't compile
	[assembly: TypeMap<ObjectiveCTypeUniverse> ("MyInnerClass", typeof (MyClass.MyInnerClass))]
    class MyInnerClass : NSObject {}
}

I don't think this would be a problem in IL though, which is probably what we're going to do anyways (emit IL instead of C# code).

[Sergio0694]

it might be better to do [...]

If we want to do that (I'm not opposed), at that point can we just use a span too for the external type mapping? 😄

public static bool TryGetExternalTypeMapping<TTypeUniverse>(ReadOnlySpan<char> id, [NotNullWhen(returnValue: true)] out Type type)
    where TTypeUniverse : ITypeMapUniverse;

I understand the lookup might still be expensive, but this would still feel like a small free perf win to me. It would've been different if we had wanted to return a dictionary, in which case, sure. But if we want to expose lookup methods directly, why not?

"I think the general rule we'll need to end up with is "if you can obtain a System.Type of the type at runtime, it should also be considered used in a cast"

Mmmhhh... I'm kinda starting to worry that putting everything together this might actually end up basically just rooting pretty much almost everything. I suppose the only way to actually find out is to actually try things out for real with a CsWinRT prototype and see what happens, and possibly go from there and see what could be improved. Hopefully it's not actually that bad 😅

"I think I see a potential problem, where the assembly attributes can't see nested private types for any typeof expressions"

You'll likely need to have an analyzer in your interop layer emitting an error for such cases, and telling users that they need their types needing marshalling to be at least accessible to the assembly. Eg. I do the same in ComputeSharp for a similar reason.

[agocke]

I shared some feedback offline, I'll copy it here as well.

I have the same concern as @MichalStrehovsky above -- now that we've moved towards generating a universal implementation using reflection, I don't think producing an IReadOnlyDictionary is helpful, and it may be harmful because it exposes more of the implementation than we want.

I also think the combination of generics and reflection is a little awkward. We should consider passing ITypeMapUniverse as either a System.Type or even just a string. A typeof in an attribute is just a string anyway.

[jkotas]

now that we've moved towards generating a universal implementation using reflection

My understanding that generation of universal implementation using reflection is not part of the current proposal.

• If the type map is generated at build time by the AOT compiler, the TryGet... APIs return true and the type map IDictionary.
• If the type map is not generated at build time by the AOT compiler, the TryGet... APIs return false and the specific interop runtime has to use some fallback solution. This fallback is not provided by the runtime.

@AaronRobinsonMSFT Is this correct?

[jkotas]

Returning an IReadOnlyDictionary makes it possible for someone to write code that iterates it or counts things in it

We do not have to implement all methods on IReadOnlyDictionary. It is fine to implement IReadOnlyDictionary methods that we do not like as throw NotSupportedException.