[API Proposal]: Allow the user to provide state for a ComWrappers.GetOrCreateObjectForComInstance operation

[jkoritzinsky]

Background and motivation

As part of the Interop Type Mapping Proposal, one important aspect we've discussed for foreign language projections to improve their performance (and avoid relying on the type map in the majority of cases), is to use call-site information about types to avoid needing to go through the map for strongly-typed APIs.
 
Currently, there is no mechanism to pass down additional state to ComWrappers.CreateObject, so any foreign language projection based on COM (ie. CsWinRT) must use something else such as [ThreadStatic] statics on their ComWrappers implementation to pass down call-site state as a side channel.

Additionally, CsWinRT has had to add additional tracking and lifetime extensions to support some of their primitives that should be immediately unwrapped, ie. IReference<T> implementations, as mentioned in #113591.

In #113591, an API change is proposed to allow CsWinRT to use another side-channel to return an object without going through ComWrappers. In the comments, a protected property is proposed to provide operation-scoped state.

This API proposal aims to solve both of the above problems. It allows the user to provide their own "state" object that will be available in CreateObjectWithState and allows the user to specify flags based on the created wrapper object with the wrapperFlags out parameter.

API Proposal

namespace System.Runtime.InteropServices;

+[Flags]
+public enum CreatedWrapperFlags
+{
+    None = 0,
+    // Same as CreateObjectFlags.TrackerObject, but decided based on inspecting the COM object directly while creating the wrapper.
+    TrackerObject = 0x1,
+    /// <summary>
+    /// The managed object doesn't keep the native object alive. It represents an equivalent value.
+    /// </summary>
+    /// <remarks>
+    /// Using this flag results in the following changes:
+    /// <see cref="ComWrappers.TryGetComInstance" /> will return <c>false</c> for the returned object.
+    /// The features provided by the <see cref="CreateObjectFlags.TrackerObject" /> flag will be disabled.
+    /// Integration between <see cref="System.WeakReference" /> and the returned object via the native <c>IWeakReferenceSource</c> interface will not work.
+    /// <see cref="CreateObjectFlags.UniqueInstance" /> behavior is implied.
+    /// Diagnostics tooling support to unwrap objects returned by `CreateObject` will not see this object as a wrapper.
+    /// The same object can be returned from `CreateObject` wrapping different COM objects.
+    /// </remarks>
+    NonWrapping = 0x2
+}

public abstract class ComWrappers
{
   protected abstract object? CreateObject(IntPtr externalComObject, CreateObjectFlags flags);
+    protected virtual object? CreateObject(IntPtr externalComObject, CreateObjectFlags flags, object? userState, out CreatedWrapperFlags wrapperFlags)
+    {
+        wrapperFlags = CreatedWrapperFlags.None;
+        return CreateObject(externalComObject, flags);
+    }

   public object GetOrCreateObjectForComInstance(IntPtr externalComObject, CreateObjectFlags flags);
+    public object GetOrCreateObjectForComInstance(IntPtr externalComObject, CreateObjectFlags flags, object? userState);
}

API Usage

class WinRTComWrappers  : ComWrappers
{
    protected override object? CreateObject(IntPtr externalComObject, ref CreateObjectFlags flags, object? userState, out CreatedWrapperFlags wrapperFlags)
    {
        if (CheckIfShouldCreateAnRcw(...))
        {
            wrapperFlags = CreatedWrapperFlags.None;
            return rcw;
        }

        // Otherwise, unbox directly
        wrapperFlags = CreatedWrapperFlags.NonWrapping;
        return UnboxTheValue(...);
    }
}

Alternative Designs

The designs in #113581 and #113591 are alternative options we've considered.

Risks

These API changes would make it less obvious which members on ComWrappers to implement in subclasses, as there are now two overloads of CreateObject.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

[AaronRobinsonMSFT]

Currently, there is no mechanism to pass down additional state to ComWrappers.CreateObject, so any foreign language projection based on COM (ie. CsWinRT) must use something else such as [ThreadStatic] statics on their ComWrappers implementation to pass down call-site state as a side channel.

What is an example of state that is needed? Are their specific downside risks or performance issues with the ThreadStatic approach?

[AaronRobinsonMSFT]

+    /// <summary>
+    /// The managed object doesn't keep the native object alive. It represents an equivalent value.
+    /// </summary>
+    NonWrapping = 0x10

I'm not sure what this does in practice. Can you please update the summary to describe what users can expect from ComWrappers? I assume that this means we won't be placing it in the cache. If so, then I think it might be better to state that, SkipCache?, rather than describe what the CreateObjectWithState() implementation is doing.

protected virtual

This change means users will have a very different implementing experience. At present, since it is abstract, the editor will provide implementations and dictate what needs to be implemented. Changing this model means it won't be obvious which APIs to implement or even that one of the CreateObject versions needs to be implemented. Perhaps this is okay, but does create another annoying barrier for an already complex interop type.

[AaronRobinsonMSFT]

This change means users will have a very different implementing experience. At present, since it is abstract, the editor will provide implementations and dictate what needs to be implemented.

I think in this case I would prefer if we didn't change the signature for CreateObject. I'd like to keep the current editor behavior as-is. This will avoid any confusion about what is expected to be written by the user and if they need the state the documentation will clearly define they should override CreateObjectWithState() in their implementation.

[AaronRobinsonMSFT]

NonWrapping = 0x10

We should also error out if this is an input value to any ComWrappers call. It is only an out for the state scenario. Open to hearing there are cases where this isn't true.

[jkoritzinsky]

I've updated the proposal above with the list of behavior changes this API would cause. I'll include them here as well:

• ComWrappers.TryGetComInstance will return false for the returned object.
• The features provided by the TrackerObject flag will be disabled.
• WeakReference interop won't work for the returned object.
• UniqueInstance behavior is implied.
• Diagnostics tooling support to unwrap objects returned by CreateObject will not see this object as a wrapper.
• The same object can be returned from CreateObject wrapping different COM objects

This is primarily to ensure that the following scenarios don't occur:

• ComWrappers.TryGetComInstance gives back a pointer to an already released (and possibly re-allocated) COM object.
• Unwrapping a COM object to either a string or System.Type that's cached in the runtime doesn't block other values from being unwrapped to those instances.

I'm fine with CreateObject continuing to be abstract and adding a new virtual instead, so I've updated the proposal.