Can we add a way to add attributes without requiring a dependency on their assembly?

[poizan42]

The requirement of depending on the assembly an attribute is defined in at runtime is a pretty common annoyance I'm encountering. A common scenario is with model/dto libraries where I want to support both NJ and STJ. If I want to say some fields should be ignored by the serializers then I need to add System.Text.Json.Serialization.JsonIgnoreAttribute and Newtonsoft.Json.JsonIgnoreAttribute. This means the model library suddenly needs a reference to both Newtonsoft.Json and System.Text.Json even though I'm only using them for metadata that's only relevant if a consumer uses one of those libraries.

I think we could use something akin to ELF's weak symbols.

Tentative proposal

As Butler Lampson put it "All problems in computer science can be solved by another level of indirection" - so that is what I'm going to propose here. Add a WeakCustomAttributeReferenceAttribute that embeds the actual attribute.

using System.Diagnostics.CodeAnalysis;
using System.Reflection;
using System.Runtime.Loader;

namespace System.Runtime.CompilerServices;

[AttributeUsage(AttributeTargets.All, AllowMultiple = true)]
public class WeakCustomAttributeReferenceAttribute : Attribute
{
    public string AttributeType { get; }
    public byte[] CustomAttributeDataBlob { get; }
    /// <summary>
    /// Initializes a new instance of the <see cref="WeakCustomAttributeReferenceAttribute"/> class with the specified target attribute type and custom attribute blob.
    /// </summary>
    /// <param name="attributeType">The canonical type name of the embedded attribute.
    /// This is either the Assembly Qualified Name, or the Fully Qualified Name if the type is present in the system library.</param>
    /// <param name="customAttributeDataBlob">A CustomAttrib blob as specified in ECMA-335 Section II.23.3</param>
    public WeakCustomAttributeReferenceAttribute(string attributeType, byte[] customAttributeDataBlob);

    public bool TryGetCustomAttributeData([NotNullWhen(true)] out CustomAttributeData? customAttributeData)
        => TryGetCustomAttributeData(AssemblyLoadContext.Default, out customAttributeData);
    public bool TryGetCustomAttributeData(AssemblyLoadContext assemblyLoadContext, [NotNullWhen(true)] out CustomAttributeData? customAttributeData);
    public bool TryGetCustomAttribute([NotNullWhen(true)] out Attribute? attribute);
    public bool TryGetCustomAttribute<T>([NotNullWhen(true)] out T? attribute)
        where T : Attribute;
}

This would obviously need compiler support so you don't have to create the CustomAttrib blob by hand - also the actual AttributeUsage of the target attribute should be enforced by the compiler. I don't know if it should have methods for inspecting the custom attribute when you don't have the types available. I guess you could use BlobReader or Mono.Cecil for that.

For this to be of much use the behavior of all the GetCustomAttribute(s) methods in System.Reflection should be updated.

• .GetCustomAttribute(s) and .GetCustomAttributesData will try to get the attribute in WeakCustomAttributeReferenceAttribute attributes. If the type successfully resolves they will act just like if the target attribute had been there directly. On the other hand if the type fails to resolve they will just completely ignore the WeakCustomAttributeReferenceAttribute.
• Overloads should be added to .GetCustomAttribute(s) and .GetCustomAttributesData that will override this behavior so you can get the WeakCustomAttributeReferenceAttribute instances and inspect them manually.
• Requesting WeakCustomAttributeReferenceAttribute directly with .GetCustomAttribute() or GetCustomAttributes() overloads taking an attribute type will return the WeakCustomAttributeReferenceAttribute as-is.

[jkotas]

It is possible to do this today by recognizing the attribute by name instead by identity. Number of attributes and libraries work like that today.

One example from many, nullability related attributes. You can see how https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Reflection/NullabilityInfoContext.cs recognizes them by name.

[poizan42]

@jkotas Unfortunately this doesn't help you if the library you are using doesn't use this pattern . Also it's just way more convenient to be able to just get your attribute with GetCustomAttribute<T>() rather than manually matching type names from CustomAttributeData and fetching arguments out of it.

Also, I just realized that my suggested implementation is missing the constructor signature. Unfortunately we can't just embed a MethodRefSig blob since it contains references to other metadata tables. I don't know if there is any common method signature serialization formats besides just using a string with ILASM syntax? I guess we could just use a Type[] array for arguments and just decide to not support attributes with constructors with weird signatures.

[jkotas]

Sure, I have been trying to point out that what you are proposing is almost achievable today. It is better to tweak what exists to address the missing parts instead of trying to invent a new parallel scheme for encoding custom attributes from scratch.

Deleting the dependency after the compilation (ie leaving the dependency reference orphaned) is another existing way to achieve the sort of thing that you are proposing. Orphaned dependencies are not unusual in the libraries out there. Even our own .NET runtime libraries ship with orphaned dependencies in .NET Framework compat shims.

The only small nuance is FileLoadException when resolving the attribute from orphaned dependency that is not present at runtime. This can be solved by introducing an attribute or an API that ignore the attributes from orphaned dependencies. No need to invent a clone of custom attribute encoding, teaching all compilers and tools how to encode and decode them, etc.

[poizan42]

A bit of brainstorming here.

"definer" is where the attribute class resides. "consumer" is code inspecting metadata for the presence of the attribute. "appliee" is anywhere the attribute is applied to.

1. No changes should be necessary to consumers
2. No changes should be necessary to definers
3. The feature should work transparently with older runtime. I.e. it should be possible to annotate attribute usages in a .net standard 2.0 library

The behavior of existing overloads of GetCustomAttribute[s[Data]] cannot be changed for existing attribute as that would be a breaking change. Adding overloads to change the behavior would not satisfy point 1.

So it's the application of the attribute that needs to be annotated somehow. Changes to the metadata format are ruled out by point 3 - even backwards compatible changes are probably a no-go since existing tooling wouldn't know the significance of the metadata. Embedding the attribute inside another attribute seems to be ruled out.

Custom attribute applications cannot have modifiers in the ECMA-335 format as far as I can tell. Applying an attribute to an attribute application is explicitly ruled out in section II.22.10.

Are custom attributes considered ordered? If they are then we could add an attribute that modifies the following attribute, e.g. something like

public class Foo()
{
    [AttributeIsWeak, System.Text.Json.Serialization.JsonIgnore]
    public int Ignored { get; set; }
}

But I can't find anywhere saying that the metadata order of custom attributes has any significance.

Maybe it would make more sense for all applications of an attribute in an assembly to be considered "weak". So something like

[assembly: AttributeIsWeaklyReferenced(typeof(System.Text.Json.Serialization.JsonIgnore))]

[KalleOlaviNiemitalo]

The only small nuance is FileLoadException when resolving the attribute from orphaned dependency that is not present at runtime.

It can even be thrown by Exception.ToString() when StackTrace.ToString attempts to recognize async functions (dotnet/coreclr#14655).

Demo

Ghost/Ghost.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <LangVersion>7.0</LangVersion>
  </PropertyGroup>

</Project>

Ghost/MissingAttribute.cs

using System;

namespace Ghost
{
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct)]
    public sealed class MissingAttribute : Attribute
    {
    }
}

Executable/Executable.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>net8.0;net48</TargetFrameworks>
    <LangVersion>7.0</LangVersion>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="../Ghost/Ghost.csproj" Private="false" />
  </ItemGroup>

</Project>

Executable/Program.cs

using System;
using System.Runtime.CompilerServices;

namespace Executable
{
    [Ghost.Missing]
    class Program
    {
        static void Main()
        {
            throw new ApplicationException("Let's see the stack trace.");
        }
    }
}

Stack trace lost on net8.0

$ dotnet run --project=Executable --framework=net8.0
Unhandled exception.
   Cannot print exception string because Exception.ToString() failed.

Stack trace shown on net48

$ dotnet run --project=Executable --framework=net48

Unhandled Exception: System.ApplicationException: Let's see the stack trace.
   at Executable.Program.Main() in C:\[REDACTED]\Executable\Program.cs:line 11

[jkotas]

Right, some tools and APIs work poorly in the presence of orphaned decencies.

[steveharter]

I do like the idea of being able to ignore exceptions (TypeLoadException, FileNotFoundException) for the attribute types that can't be resolved so that dangling type references can be supported.

However, I'm not sure it meets the bar for a new feature. The simple JSON scenario mentioned makes sense, but as soon as you need a custom converter or use other serializer extension points within the DTO library then the benefit is lost.

Without adding new features to the core libraries or runtime, the "appliee" which owns the DTOs could use a feature switch that would differentiate between STJ and Newtonsoft. During trimming then, only one set of functionality would apply and the unreferenced attributes should go away.

However, the "consumer" would need to know about the feature switch. If the consumer is already publishing to reduce trim size, that doesn't seem to be a blocking requirement. Perhaps we should make it easier for a library to expose which trimmer flags it uses (such as through an assembly attribute) so the consumer can see these in Visual Studio for example?

But if we did want to go down the "ignore exceptions" path with only the "appliee" knowing about it, the assembly-level approach may work:

[assembly: IgnoreTypeLoadExceptions(nameof(System.Text.Json.Serialization.JsonIgnoreAttribute))]
[assembly: IgnoreTypeLoadExceptions(nameof(Newtonsoft.Json.JsonIgnoreAttribute))]

class DTO
{
    [System.Text.Json.Serialization.JsonIgnore]
    [Newtonsoft.Json.JsonIgnore]
    public int MyProperty { get; set; }
}

along with runtime support to ignore the exception(s). The attribute would probably be declared like:

namespace System.Runtime.InteropServices; // or System if the compiler doesn't need to see it
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public class IgnoreTypeLoadExceptionsAttribute : Attribute
{
    public string TypeName { get; }
    public IgnoreTypeLoadExceptionsAttribute(string typeName)
    {
         // If type is not an attribute, throw here
        TypeName = typeName;
    }
}