[Proposal]: Extension interface implementation

[agocke]

Discussion for proposal #9319

[agocke]

Responding to one of the comments from @HaloFour

These rules sound very similar to the coherence rules enforced by Rust. My opinion is that such limitations are fatally limited in that I expect that the vast, vast, vast majority of use cases expected by users of a feature like this would be very precisely to extend a type nor declared in the current project with an interface also not declared in the current project.

Rust's rules are called orphan rules and they are very similar to the rules that I listed in the linked discussion. However, the motivation is not the same, and it turns out this is closer to "coincident evolution" than a mirroring of ideas.

Rust has orphan rules primarily because they want to guarantee particular notions of consistency in their type system and avoid some challenges with typing in a system where multiple implementations can "cross" each other.

In contrast, the orphan rule I propose here is mainly about implementation choices. Since the very beginning of .NET there have been two things that have been built heavily into the runtime: interface implementation is part of type identity, and type identity is fixed. Changing either of these things is almost impossible in .NET. You start messing with type identity and things blow up. You start trying to separate interface implementation from type identity, things blow up.

What the orphan rule brings to .NET is the ability to adjust the interface implementation list without changing type identity. Because everything is coherent and entirely in the view of the runtime at type load time, the runtime can effectively shim any set of interface implementations into existing types without changing identity. That's not to say this is an easy task, but it's at least understandable. This would still be an incredibly tricky change to the runtime, but I think it's possible.

Absent an orphan rule, I'm skeptical that any alternative is going to progress beyond brainstorming. For anyone who is interested in tackling that challenge: you need to start at the ECMA-335 level. Your task will be to walk every single rule regarding type identity, conversions, interface implementation, interface dispatch, generics, and reflection, and describe precisely how they will behave in your design. At minimum, I expect you will need to answer the following problem: List<T> where T doesn't implement some interface, List<T> where T does implement some interface, and List<T> where T does implement some interface, but with a different implementation, must all coexist at runtime. T will nominally appear to be the same type. List<T> will also need to sometimes be the same type and sometimes not. You cannot modify List<T> itself in any way, as the code is already written. How will your system handle all of those circumstances?

[HaloFour]

To bring my two comments over for context: 😄

These rules sound very similar to the coherence rules enforced by Rust. My opinion is that such limitations are fatally limited in that I expect that the vast, vast, vast majority of use cases expected by users of a feature like this would be very precisely to extend a type nor declared in the current project with an interface also not declared in the current project.

My other issue with a design like this is that by impacting the type at runtime you lose all of the scoping mechanisms that exist for extensions in general. IMO, "extension implementation" should have similar scoping rules, in that I can influence them based on the namespaces/classes imported in that compilation unit. Obviously that requires the compiler to be involved in the design as the runtime would have no concept of that.

I've had a number of conversations very recently about what I think are the major strategies for implementing something like "extension implementation" which is why this is very front of mind. 😄

Absent an orphan rule, I'm skeptical that any alternative is going to progress beyond brainstorming.

That depends on what the goals are. If the goal is to affect the runtime so that the target type always appears to implement said interface, sure, these rules are necessary. But I don't think that is the only approach.

Take Scala implicit classes, for example. They allow you to bridge any arbitrary type to any arbitrary trait through a witness class that is the implicit class. This is done entirely by the compiler, and has scoping behavior that is practically identical to extension methods. As long as the implicit class is in scope, you can effectively treat the target type as if it implements that interface. To me, this is the behavior I would expect from something that aligned itself with "extensions".

I think a role-based approach would work very differently, too, especially if a "role" was a kind of type that the runtime understood and supported. Being very hand-wavey (both because I haven't thought about it a huge amount and because I don't know nearly enough about the inner workings of the runtime) but I would envision a special type that appeared to extend the receiver type (even though it might not be possible due to the receiver type being sealed, a struct, etc.), that would also extend from the various interfaces. That "role" type would be reflected in the metadata, but it would end up elided away by the runtime and virtual calls to the interface would be patched up to the extension implementation declared as a part of that role.

Implementation details aside, I think it's much more important to focus on the use cases that users actually want to solve. Any use case I've seen here involved a BCL type and a BCL interface. I was talking to someone on the Discord channel just a couple of days ago who wanted the ability to make arbitrary BCL types appear to implement IEnumerable<T> or IEquatable<T>. None of those use cases can be solved by an implementation that requires coherence an orphan rule to ensure safety.

[HaloFour]

To be very clear, I am not arguing against your proposal. I think it makes a lot of sense and I agree that the orphan rule is probably necessary. I would expect that it would be the only real practical way to attach interfaces to an external type from a runtime perspective. 😄

[agocke]

Take Scala implicit classes, for example

Yeah I think this is a good alternative solution. Unfortunately I’m also a bit skeptical that the kind of reflection-heavy code that .NET users write would play well in that world. But it’s an alternate design point (although worth mentioning that it’s controversial even within scala).

Implementation details aside, I think it's much more important to focus on the use cases that users actually want to solve

I don’t really buy this argument. If there isn’t an implementation strategy that can easily solve all possible requirements, the implementation has to play a role in the discussion. Fundamentally we will always be constrained by what’s practical. This is engineering, after all.

[HaloFour]

Yeah I think this is a good alternative solution. Unfortunately I’m also a bit skeptical that the kind of reflection-heavy code that .NET users write would play well in that world.

Totally agree. Scala's approach works really well when you want to quickly treat a type like a trait, but it doesn't help with reflection or even basic type checking. It's definitely more of an implicit one-way conversion between the receiver type and the target interface. I have no idea if any kind of runtime support could help here.

I don’t really buy this argument. If there isn’t an implementation strategy that can easily solve all possible requirements, the implementation has to play a role in the discussion. Fundamentally we will always be constrained by what’s practical. This is engineering, after all.

Yeah, that makes sense. At least go down the rabbit holes enough to understand the pros and cons of each approach and which use cases they can potentially solve. I really don't think there is an obvious "winner" here, and it's possible that two (or more) approaches could be adopted side by side.

[iam3yal]

@HaloFour, @agocke I get the argument that it wouldn't play nice with the existing reflection APIs but it can be designed in such a way where it's reflection friendly and maybe based on use-cases even extend the existing reflection APIs... again it all depends on the goals here and maybe again depends on the design, reflection support can come later? Just my 2c.

[fabianoliver]

I like this proposal.

Massively subjective of course: Personally, I don't seem to recall any case where I would've needed or liked to implement some third party interface for some third party type. If I ever had to, I'd imagine that effectively wrapping objects into some decorating type, either statically or via dynamic proxy and mixing in some new interface implementation would typically get the job done?

Similarly, I'm not a huge fan of the idea of scope-bound extension interface implementation just yet. Makes code harder to follow (granted, same can be said about extension methods). And too many question marks for my liking.. say you have a function taking a parameter of type object. Would a "obj is ISomeExtensionInterface" check succeed if the caller of said function had the extension in scope? If the extension is in scope for the function itself? What if the type check was reflection based?
There are undoubtedly clear answers to all of that, but it doesn't seem like something that's easy to intuit, which IMHO is a big negative for any feature (Or maybe I'm missing something fundamental and those are non-issues?!)

Anyways, back to this proposal here: Conditional interface implementation would be quite nice to have. Especially for functional code - think of option or result types that could implement iequatable, icomparable etc if and only if their inner type does as well.
Is that worth a complex change to the runtime? I can't judge that, but I could've definitely used that feature on occasion. And as of now, I unfortunately don't think there's much of a way to fudge your way around it's absence easily.

For understanding, could the orphan rule be relaxed further? For example, could you allow an extension for an external interface of some external type if it's defined in the application's entry assembly - I'd imagine maybe that might be scannable before implementing type or interface is loaded?

[julealgon]

If I ever had to, I'd imagine that effectively wrapping objects into some decorating type, either statically or via dynamic proxy and mixing in some new interface implementation would typically get the job done?

But this proposal is precisely so you don't have to create custom wrappers/decorators though.

It is similar to extension methods. You don't need them: you can just define a static helper class and call the methods there. But the point is how to get that same behavior with a simpler, more intuitive developer experience.

Not only that, but depending on how this is implemented, it could also be more performant than leaving it to you to create a wrapper. It is a big win when the runtime/compiler can create a very optimized pattern and let you just tap into it, instead of having developers come up with their own sub-optimal approach to the same problem.

[HaloFour]

Yes, this proposal would likely be more performant and seamless in the .NET ecosystem, in that reflection, type checking and everything will "Just Work" as if the underlying type had actually been written to implement that interface. However, that comes with the caveat that the runtime must ensure that there can only ever be one implementation between an interface and a type, and that's where the orphan rules come into play. So while it is a much more seamless experience, it can only ever work for interfaces that you declare within your own project. And those interfaces cannot extend from interfaces that are referenced from other assemblies either.