Produce SDK-only archive file without shared frameworks

[mthalman]

Currently the SDK is shipped as a monolithic archive file that includes the SDK and shared frameworks (Microsoft.NETCore.App, Microsoft.AspNetCore.App, Microsoft.WindowsDesktop.App). But there is a need for the ability to construct a .NET installation piecemeal.

A scenario where this would be used is with Docker images. As part of the changes in dotnet/dotnet-docker#1351 and dotnet/dotnet-docker#1814, the SDK image will now be based on the ASP.NET image, which itself is based on the runtime image. This kind of layering means that each framework will be installed individually. The runtime image installs Microsoft.NETCore.App and dotnet.exe. The ASP.NET image builds on top of the runtime image by also installing Microsoft.AspNetCore.App. The SDK image builds on top of the ASP.NET image by also installing the SDK and Microsoft.WindowsDesktop.App. This layered approach requires that the SDK and frameworks be made available as separate archive files so they can be retrieved individually.

Another scenario is the concept of a single SDK that can be shipped with multiple runtime versions.

Both scenarios are discussed here: dotnet/dotnet-docker#1482 (comment).

Without this kind of componentization of the SDK archive, it requires implementers of piecemeal installation to have very specific knowledge of the .NET installation folder structure so that it only pulls out the components it needs. For example, the command to retrieve just the SDK-only components from the SDK archive file would look something like this: tar -C /usr/share/dotnet -oxzf dotnet.tar.gz ./packs ./sdk ./templates. This kind of dependency with consumers is fragile and can easily lead to breakage if changes are made to the folder structure.

The proposal is to produce an archive file of the SDK that contains only the components that are SDK-specific and none of the shared frameworks. This would essentially consist of the following folders: packs, sdk, templates.

Another thing to consider is LICENSE.TXT and ThirdPartyNotices.txt, related to dotnet/installer#7043, which should probably be moved into a relevant sub-folder.

This work should also be done in conjunction with https://github.com/dotnet/wpf/issues/2865. Without the WindowsDesktop package being available as a separate archive file, having an SDK-only archive file doesn't provide much benefit for Windows scenarios.

cc @MichaelSimons, @richlander @mmitche

[mthalman]

@marcpopMSFT - Just wanted to make you aware that we've now authored and shipped Docker images that expose the unfortunate need to be selective about what is pulled out of the SDK archive file. Example: https://github.com/dotnet/dotnet-docker/blob/86586d6f1823014ae033513d5dce4130d8fe4ce1/5.0/sdk/alpine3.11/amd64/Dockerfile#L30

It would be ideal if this could be implemented for 5.0 to avoid the need for this kind of hacky workaround.

[marcpopMSFT]

We are planning on looking at the installation of the sdk in the .net 6 timeframe and this request seems to dovetail with those plans. @wli3 and @KathleenDollard are working on those plans. .net 5 looks pretty tight at the moment for us so I don't think we can take on additional work though we'll be revisiting our plans in a month or too based on what progress we've made to our committed work.

[wli3]

Although I'd like to understand why "the SDK image will now be based on the ASP.NET image, which itself is based on the runtime image".

The SDK zip should have an aspnet runtime and a base runtime. Why we cannot base the image on sdk zip?

Also, we don't test against arbitrary runtime with a certain SDK at all. And it is only tested against one runtime which is in the bundle.

[mthalman]

Although I'd like to understand why "the SDK image will now be based on the ASP.NET image, which itself is based on the runtime image".

The SDK zip should have an aspnet runtime and a base runtime. Why we cannot base the image on sdk zip?

We can. It's just not ideal to do that because it requires hard-coding to get the SDK-specific components out of the sdk zip file. If the contents or folder structure changes, then this could lead to a breaking change for consumers, such as our Dockerfiles, that need to do this sort of thing.

Also, we don't test against arbitrary runtime with a certain SDK at all. And it is only tested against one runtime which is in the bundle.

All that's being said here is that this sort of componentization would help to support such a scenario. There's nothing saying that it needs to be supported. There has been discussion of this scenario though.

[wli3]

We can. It's just not ideal to do that because it requires hard-coding to get the SDK-specific components out of the sdk zip file.

What component do you need to remove? SDK need the base runtime to function, so the only thing can be removed is aspnet runtime.

this sort of componentization would help to support such a scenario.

That depends. Currently, SDK has a hard dependency on the latest runtime(the time it ships). And aspnet runtime version need to be in sync with base runtime to avoid dll hell. For 6.0, we plan to make aspnet runtime separately installable. But I think we will still bundle a base runtime. That goes back to my question about, why do you need to separate out the base runtime?

[mthalman]

What component do you need to remove? SDK need the base runtime to function, so the only thing can be removed is aspnet runtime.

The hierarchy of the Docker images is this:
Runtime <- ASP.NET <- SDK

The runtime image only installs the runtime bits. The ASP.NET image only installs the ASP.NET bits but it "inherits" the runtime bits from the runtime base image. The SDK image only installs the SDK bits but it "inherits" the ASP.NET and runtime bits from its base image. So the runtime bits are there, we just don't need the runtime bits contained in the SDK zip. All the build versions are in sync because of how each base image is referenced.