Eliminate the need to bootstrap new major SDK versions

[lbussell]

Describe the Problem

Linux distros need to bootstrap each new major versions of .NET.
Bootstrapping takes a considerable amount of time and effort, sometimes requiring special approval or overrides from distros.

Describe the Solution

Ideally, we should be able to build new major versions of .NET using the previous major version. For example, building .NET 8.0.100 with the latest source-built version of .NET 7.0.

This imposes a significant restriction on .NET product teams, as they would not be able to use new .NET features until they are generally available in a public source-built release.

A potential solution is building a (small) bootstrap compiler against the previous .NET SDK, which is capable of building the current version of Roslyn, which can then be used to build the whole SDK.

cc @mirespace @omajid @richlander @crummel @MichaelSimons

[omajid]

cc @ayakael who has multiple stages of builds to get .NET building in Alpine.

[MichaelSimons]

cc @dotnet/distro-maintainers, @mmitche

[tmds]

A potential solution is building a (small) bootstrap compiler against the previous .NET SDK, which is capable of building the current version of Roslyn, which can then be used to build the whole SDK.

We're shifting the problem to this compiler.

Bootstrapping takes a considerable amount of time and effort, sometimes requiring special approval or overrides from distros.

Afaik, for a bootstrap build, you need to build twice to avoid having used prebuilts.
That cost you some build time.

At that point, your artifacts are just the same as a non-bootstrap build.
So, why does the bootstrap build require extra approval?

[MichaelSimons]

[Triage] @mirespace, @ayakael - We would be interested in hearing your perspective on the impact of this issue. How much pain is this causing for you? TIA

[tmds]

At that point, your artifacts are just the same as a non-bootstrap build.
So, why does the bootstrap build require extra approval?

Answering my own question: the approval is needed for storing the prebuilts that are are consumed by the bootstrap build (it isn't about the artifacts that are produced).

Afaik, the repos (like runtime, and aspnetcore) adopt new features as they become available from the preview releases. That means the preview 2 build of a repo adopts features delivered by the preview 1 sdk. And the preview 1 builds get built using the previous major release's sdk (final or rc).
So if you build all the previews and rcs, you could avoid bootstrapping. But then again, source-build needs to support building all these previews, and a maintainer needs to build them too.

The repos depend on an sdk version (not just the compiler).
So the repos for .NET X+1 expect an SDK with the feature set of X+1. I don't see us patching up a source-build .NET X sdk so it matches that expectation.

[MichaelSimons]

Afaik, the repos (like runtime, and aspnetcore) adopt new features as they become available from the preview releases. That means the preview 2 build of a repo adopts features delivered by the preview 1 sdk. And the preview 1 builds get built using the previous major release's sdk (final or rc).

Yes, repos often take dependencies on new features. Today they can take a dependency before it is even released via a preview. I think we could get a policy approved stating teams can only take a dependency once it has been released via preview. Limiting teams to use the N-1 toolset likely wouldn't fly and neither would backporting/patching.

But then again, source-build needs to support building all these previews, and a maintainer needs to build them too.

Agreed, this is a goal of the source-build team going forward.

[vorlonofportland]

At that point, your artifacts are just the same as a non-bootstrap build. So, why does the bootstrap build require extra approval?

Ubuntu is a self-hosting open source operating system and we expect all our source packages to build against our own binaries. Bootstrap builds are exceptional and, due to the security implications of being able to inject arbitrary third-party binaries into the build process, are restricted to a small number of Ubuntu developers and are thus comparatively costly for us. We would of course have no problem trusting the Microsoft binaries for such a bootstrap, but it would still be our preference to not have to rebootstrap the same language compiler from binaries repeatedly across major versions. (For almost all self-hosting compilers we carry in Ubuntu, the new version can be built using the previous version; so with few exceptions a bootstrap is a one-time thing.)

[ayakael]

As @omajid has eluded to, we've built a multistage building system on Alpine that depends on Microsoft's prebuilt SDK to create our locally hosted build of dotnet. Our build pipeline for every new release of dotnet and Alpine depends on Microsoft prebuilt SDks to build an initial tainted stage0 SDK that we then package and use to build our initial self-hosted dotnet packages. Because we at Alpine aim to rebuild from scratch our self-hosted compilers at every Alpine release, we maintain our stage0 package as to support that. After the initial build of our main dotnet package, future builds uses our main bootstrap packages rather than stage0

Thus eliminating the need to bootstrap new major SDK versions would be nice, but would ultimately not change how we do things at Alpine. If we could build dotnet7 with dotnet6, that would be nice. But given dotnet7 actually needing dotnet7-rcx to build, this doesn't look to be that useful.

[ayakael]

My mistake, I reread the proposal and the mini compiler idea seems quite interesting. This would actually fit quite nicely in our stage0 implementation. As it is now, we already build a minimum set of components (runtime, roslyn, sdk, aspnetcore, installer) for building an SDK tarball. stage0 could thus pull the previous dotnet7 bootstrap compiler instead of upstream Microsoft tarballs to build dotnet8-stage0. dotnet8-build would then take the dotnet8-stage0 artifacts to build the first proper dotnet8 package.

[mmitche]

As @tmds mentioned, I think we're likely to hit issues with teams taking dependencies on non-compiler SDK features prior to P1. We might be able to get a concession from the teams there, but it may be tougher sell. I think it's worth a discussion though, @MichaelSimons.

[MichaelSimons]

As eluded to earlier, we have received pushback from the repo teams and this is not feasible for us for .NET 8.0. The Unified Build investments will make the situation better and once done, we will have to re-evaluate this.