Consider deprecating older source generators and their associated infrastructure

[ViktorHofer]

See #82179 (comment) for the initial conversation.

Packages which include source generators are growing unboundedly because they not just contain assemblies targeting the current roslyn but also carry along all the previous targeted source generators, i.e. roslyn3.11, roslyn4.0 and roslyn4.4. Those never get deleted because the package could be loaded in an environment (SDK / Visual Studio) that doesn't support the newer version of the compiler.

That effectively means that we need to support our roslyn3.11 source generators until the minimum version of Visual Studio that supports the generators is EOL. Based on the official lifecycle document, that is April 2029.

Consequences based on that:

• Packages are growing unboundedly whenever their underlying source generators target a new version of roslyn.
• Old / legacy source generator source code can't be removed. In the example of roslyn3.11 source generators, they aren't incremental and cause performance issues.
• Custom infrastructure for polyfilling SDK features in older environments adds complexity to packages. That infrastructure is repository specific (their is no shared code today at all) and written in msbuild. A package consumer needs to pay for evaluating that custom infrastructure even on a current environment.
• Whenever a new source generator is added, we need to add new projects and grow the build graph as msbuild doesn't allow multi-targeting based on the compiler version.

Those pain points could be mitigated by applying a required toolset version to our packages. While the toolset version refers to the minimum NuGet client version, the compiler and NuGet both ship in the SDK and Visual Studio and by picking a specific NuGet version, a compiler version is enforced as well. Alternatively we could discuss if we want to add another metadata to the nuspec file that indicates the required compiler version. Downside of that is that we would need to backport that support into those very old versions of the SDK and Visual Studio.

By applying a minimum required toolset version, we could remove those old source generators, their underlying infrastructure and with that make our packages smaller and less complex. The downside in doing that is that we would prevent customers on older environments to reference the very latest versions of our packages.

cc @ericstj @eerhardt @jaredpar @stephentoub @jkotas

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

See #82179 (comment) for the initial conversation.

Packages which include source generators are growing unboundedly because they not just contain assemblies targeting the current roslyn but also carry along all the previous targeted source generators, i.e. roslyn3.11, roslyn4.0 and roslyn4.4. Those never get deleted because the package could be loaded in an environment (SDK / Visual Studio) that doesn't support the newer version of the compiler.

That effectively means that we need to support our roslyn3.11 source generators until the minimum version of Visual Studio that supports the generators is EOL. Based on the official lifecycle document, that is April 2029.

Consequences based on that:

• Packages are growing unboundedly whenever their underlying source generators target a new version of roslyn.
• Old / legacy source generator source code can't be removed. In the example of roslyn3.11 source generators, they aren't incremental and cause performance issues.
• Custom infrastructure for polyfilling SDK features in older environments adds complexity to packages. That infrastructure is repository specific (their is no shared code today at all) and written in msbuild. A package consumer needs to pay for evaluating that custom infrastructure even on a current environment.
• Whenever a new source generator is added, we need to add new projects and grow the build graph as msbuild doesn't allow multi-targeting based on the compiler version.

Those pain points could be mitigated by applying a required toolset version to our packages. While the toolset version refers to the minimum NuGet client version, the compiler and NuGet both ship in the SDK and Visual Studio and by picking a specific NuGet version, a compiler version is enforced as well. Alternatively we could discuss if we want to add another metadata to the nuspec file that indicates the required compiler version. Downside of that is that we would need to backport that support into those very old versions of the SDK and Visual Studio.

By applying a minimum required toolset version, we could remove those old source generators, their underlying infrastructure and with that make our packages smaller and less complex. The downside in doing that is that we would prevent customers on older environments to reference the very latest versions of our packages.

cc @ericstj @eerhardt @jaredpar @stephentoub @jkotas

Author:	ViktorHofer
Assignees:	-
Labels:	area-Infrastructure-libraries, discussion
Milestone:	8.0.0

[sharwell]

... In the example of roslyn3.11 source generators, they aren't incremental and cause performance issues. ...

This isn't really the case. In practice, one of two things is going to happen:

1. The source generator is being used in a non-incremental environment (e.g. build0, where ISourceGenerator and IIncrementalGenerator exhibit very similar performance characteristics.
2. The environment doesn't support something newer, in which case the generators are running as fast as is possible for the environment.

[sharwell]

Packages are growing unboundedly whenever their underlying source generators target a new version of roslyn.

Note that it's possible this has a natural upper bound. As the performance of the infrastructure is tuned to the source generators and stabilizes, we may find that newer releases of Roslyn require little to no changes to the source generators themselves. I'm concerned that by posing the question so soon after the introduction of significant features targeting runtime source generator scenarios (resulting in relatively large changes to the generator implementations), our outlook on the past year may be biasing our view of what the next 6 years look like for the same generators.

[jkotas]

our roslyn3.11 source generators until the minimum version of Visual Studio that supports the generators is EOL. Based on the official lifecycle document, that is April 2029.

.NET Core lifecycle forces you to upgrade Visual Studio much faster. Why do we need to worry about Visual Studio lifecycle?

[ViktorHofer]

While that lifecycle applies to modern .NET, packages like System.Text.Json support netstandard2.0 as well and include source generators. AFAIK, nothing hinders you from consuming an 8.0.x package on and old Visual Studio installation and target netstandard2.0.

[jkotas]

The linked PR has the tests ifdefed for netcoreapp - https://github.com/dotnet/runtime/pull/82179/files#diff-415bb466216b8b4fd07992787af62f30e8a2d6d83fff4b627b88176628db7731R14 . I assume that this source generator is not going to support netstandard2.0. Is that correct?

I expect that it is going to be common for new source generators to depend on new C# language features that will make it hard to target netstandard2.0. We do not generally support new C# language features with netstandard2.0. Can we limit source generators to netcoreapp targets only?

If we have shipped some netstandard2.0 targeting source generators already, can we tell how many people are actually using them with netstandard2.0? If the set is small, can we drop that support or require Visual Studio upgrade?

[ViktorHofer]

I assume that this source generator is not going to support netstandard2.0. Is that correct?

You mean the new one that @layomia is working on, right? He currently targets netstandard2.0: https://github.com/dotnet/runtime/pull/82179/files#diff-5cd2b9bb815efd906e5edd5f1c4bc5b1a3e5dc6e6d34195fd313e75b989c2ed5R3

When targeting modern .NET, how would Roslyn running on .NET Framework inside Visual Studio handle loading a .NETCoreApp only source generator?

[stephentoub]

I expect that it is going to be common for new source generators to depend on new C# language features that will make it hard to target netstandard2.0.

You mean the new one that @layomia is working on, right? He currently targets netstandard2.0:

There are two relevant versions with a source generator: what the source generator requires in order for itself to run, and what the code it generates requires. I believe (though he can correct me if I'm wrong) what Jan is asking about is the latter, e.g. that source generator is emitting code that uses nullable (https://github.com/dotnet/runtime/pull/82179/files#diff-eaa3c60ce61aebffabb88f108fdbb371de78fdd4b2d0526c26b24bc60e2047bbR2), which requires a C# language version of at least C# 8, but .NET Framework and .NET Standard 2.0 only officially support using language version 7.3.

[jkotas]

Yes, I am talking about the language version required by the source generator.

[jaredpar]

Why do we need to worry about Visual Studio lifecycle?

Even though the .NET lifecycle for runtimes moves faster, the lifecycle for TFMs does not in the .NET SDK. It is still legal and supported to target netcoreapp3.1 for libraries for build and design time and will be until we change the .NET SDK to put a finite lifetime on it (seems unlikely). That means by extension it's supported to do this in Visual Studio.

Once source generators enter the picture it means that the .NET SDK has design time assets that are a part of the editing experience. Those need to be setup for servicing for the lifetime of the Visual Studio they run in. How the servicing works is a bit more flexible but they do need to be setup for servicing.