Provide mapping of reference assemblies to their implementations

[tmat]

In order to improve Go To Definition in Metadata experience (dotnet/roslyn#24349) the IDE needs a way to find all implementation assemblies that have shipped for a given reference assembly.

There are to main cases to consider:
a) the reference assembly is in a package in the nuget cache
b) the assembly does not ship in a package (it’s in a reference assemblies directory in an SDK)

[a] is trivial to make work. We just walk the nuget cache directory structure starting with the reference assembly path and find the lib directory that has implementation assemblies. We gather implementation assemblies for all TFMs from there.

[b] is more complicated. What need is a map of reference assembly identity to the MVID of the latest version of implementation assembly for each supported TFM.
This map would need to be updated as the products that ship the implementation assemblies evolve (both .NET Framework and .NET Core).

The main question is how would this map be distributed. Since .NET Framework doesn’t change that often, and if so the changes usually affect only a few source files, we could ship the map with the IDE and update it every time the IDE is updated.
.NET SDK could also ship with the map and if there is a way to locate it reliably based on the location of the reference assembly.

The IDE would use this map as follows.

When the user navigates from source file to a metadata symbol that comes from a reference assembly, the IDE queries the map to find a list of all TFMs that the reference has an implementation for.
The IDE considers the project context the current source file is opened in (displayed above the text buffer on the left in a combo box in VS). Based on the project context TFM it select the most appropriate TFM the reference assembly has an implementation for and queries the Microsoft symbol server for the MVID of the implementation assembly as well as for the PDB corresponding to this implementation assembly (both MVID and PDB ID would be stored in the map). Once the implementation assembly and PDB is available the IDE finds the metadata symbol in the implementation assembly and determines what source file contains its definition based on debug information in the PDB. It then uses Source Link record present in the PDB to locate the source file on GitHub, it downloads the source file and opens it in the editor. It also fills the project context combo-box with TFMs that the reference assembly is available for and with an extra item that represents the reference assembly itself. This allows the user to switch to a different TFM, repeat the above process and see the source of the symbol definition in other implementation assemblies, or just the declaration header of the reference assembly.

[tmat]

@terrajobst

[terrajobst]

I like that, because it means that the user isn't limited to what they have on the box.

However, I think users might expect symmetry between what they see when they debug and step into vs. what they see if they F12 outside the debugger, which in this world would be different. What are your thoughts on this?

[tmat]

Yes, there are a few adjustment to the debugger that would need to be implemented.

There are a couple scenarios I can think of right now:

1. The user first uses Go To Def to navigate to the implementation assembly. Let's say it's the one for net472 and they have .NET 4.8 on their box. Now they try to place a breakpoint to this file.

If the source didn't change between 4.7.2 and 4.8 then the breakpoint would bind when the debugger attaches and loads the 4.8 implementation assembly and the user can step thru it. If the source didn't match then the debugger would show an unbound breakpoint with a message that the source file doesn't match any loaded assembly. I think this behavior is fine, but we might want to tweak it a bit to make the experience better. The debugger could offer the user to fetch the right version of the file (that corresponds to the actually loaded implementation) and move the breakpoint to it.

2. The user first uses Go To Def to navigate to the implementation assembly. The user places a breakpoint to their app code, debugs thru their code and steps into a framework method whose source happens to be in the file that GTD opened before.

Currently the debugger fetches the sources for the loaded implementation assembly via Source Link, which may be the same or different from the already opened file.
Here we would use an IDE service that makes sure the debugger is not opening a duplicate source file to the one that's already open (if the sources match) or when it opens such file the buffer is opened with the right context in the combo-box.

[dsplaisted]

There isn’t directly logic that maps from a reference assembly from a .NET targeting pack to an implementation assembly. The targeting pack has a bunch of reference assemblies in it. If an app is built self-contained, then the implementations will come from a runtime pack which also has a bunch of assemblies in it. I’m not sure there’s always going to be a one-to-one mapping though, since some of the assemblies on either side might be type forwarding facades. Plus, the runtime pack is not part of the SDK installation, it is downloaded during NuGet restore for self-contained apps. So you can’t really rely on the runtime pack being on the machine.

Given the MVID of an assembly, con you download the implementation as well as the PDB from the Microsoft symbol servers? If so, then maybe a good solution would be to put the MVID of an implementation assembly in an attribute in the reference assembly, or have some other way either in the targeting pack or the SDK itself of mapping from a reference assembly to the MVID.

[davidwengier]

can you download the implementation as well as the PDB

I don't believe implementation assemblies are on symbol servers, and there is more than just the MVID needed to find the PDB (though maybe not a lot more?). The main issue here, even if we could find the PDB and source link URL without an implementation assembly, that doesn't help the decompilation scenario, which we'd also like to fix. No getting around needing real IL for that one.

the runtime pack is not part of the SDK installation, it is downloaded during NuGet restore for self-contained apps

Is there anything we can do to help in some percentage of cases, even if not all? eg. I realise if I'm targeting .NET 5 but .NET 6 is installed I won't have the right implementation assemblies, but what if I am targeting .NET 6? Then the implementation assemblies must be somewhere, or my app wouldn't run. Is there any kind of incremental work we can do here?

[tmat]

I don't believe implementation assemblies are on symbol servers

They most definitely should be, they are needed for mini-dump analysis.

[tmat]

What we need is to establish a map between reference assembly MVID and MVIDs of all existing implementation assemblies. dotnet/runtime and .NET SDK build infrastructures should be able to generate such a map for each build and we can ship that map with Visual Studio. It would be updated every time .NET SDK inserts into VS.

Alternatively, we can investigate having all dotnet/runtime builds indexed by codex and use that to maintain the mapping.

[davidwengier]

They most definitely should be, they are needed for mini-dump analysis.

Very happy to be wrong about this :)

Do we need more than MVIDs? Currently to get a PDB from a symbol server we need to supply checksums, pdb age etc.. Or is it that MVID is enough to pull down the implementation assembly, and then we can use that as normal?