Add .NET 10 target, #1219 (#1222)

* Remove netstandard2.1 target and test support via net6.0, #1040

* Add net10.0 target

* Fix merge issue

* Regenerate github workflows; test OpenNLP on net10

* More C# 14 field keyword changes

* .NET 10 build support in Azure DevOps

* Remove net9.0 tests for lucene-cli

* Add back net9.0

* Remove net9.0 from lucene-cli

* Conditionally set IsPublishable

* Regenerate github workflows

* Remove net9.0 tests

* PR feedback

* Add .NET 9 tests to ADO

* Upgrade Antlr4BuildTasks to 12.11.0

* Fix net9.0 test build

* Revert SetTargetFramework change

* Add net9.0 tests to CLI and OpenNLP test projects

* Directory.Build.props: Added section to compare Visual Studio versions (including prerelease versions) to use to drive IsLegacyVisualStudioVersion property across the solution

* SWEEP: Added legacy support for Visual Studio 2022, excluding net10.0, netstandard2.0, and .NET Framework from compilation. net10.0 isn't supported at all. The others cause invalid warnings in VS 2022 and excluding them from compile is the simplest way around that.

* publish-test-results-for-test-projects.yml: Upload test results for net10.0 for Lucene.Net.Tests.Cli and Lucene.Net.Tests.Analysis.OpenNLP

* publish-test-results-for-test-projects.yml: Upload test results for net9.0 for Lucene.Net.Tests.Cli and Lucene.Net.Tests.Analysis.OpenNLP

* .build/runbuild.ps1 + github/workflows/Generate-TestWorkflows.ps1 + Directory.Build.targets: Added a key identifier SupportedTargetFrameworks to the MSBuild query so it can be parsed using a regex rather than relying on the position in the output (which changed in .NET 9). Updated all callers to use a Get-SupportedTargetFramworks() function to encapsulate the logic of retrieving the target frameworks and parsing out the string we are interested in with an error message if matching fails.

* .github/workflows/Generate-TestWorkflows.ps1: Generate workflow files with no BOM

* Regenerated GitHub test workflows

* Fix comment

---------

Co-authored-by: Shad Storhaug <shad@shadstorhaug.com>

Author: paulirwin

.build/azure-templates/publish-test-results-for-test-projects.yml

@@ -70,7 +70,7 @@ steps:
 #  displayName: 'Validate Template Parameters'
 
 
-# Special case: Only supports .NET Standard 1.3
+# Special case: Only supports .NET Standard 1.3 (we only need to test this on one framework because it is for VS)
 - template: publish-test-results.yml
   parameters:
     testProjectName: 'Lucene.Net.Tests.CodeAnalysis'
@@ -81,23 +81,64 @@ steps:
     testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
     testResultsFileName: '${{ parameters.testResultsFileName }}'
 
-# Special case: Only supports net8.0
+# Special case: Only supports net10.0, net9.0 and net8.0
+
 - template: publish-test-results.yml
   parameters:
     testProjectName: 'Lucene.Net.Tests.Cli'
-    framework: 'net8.0' # Since condtions are not supported for templates, we check for the file existence within publish-test-results.yml
+    framework: 'net10.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
+    vsTestPlatform: '${{ parameters.vsTestPlatform }}'
+    osName: '${{ parameters.osName }}'
+    testResultsFormat: '${{ parameters.testResultsFormat }}'
+    testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
+    testResultsFileName: '${{ parameters.testResultsFileName }}'
+
+- template: publish-test-results.yml
+  parameters:
+    testProjectName: 'Lucene.Net.Tests.Cli'
+    framework: 'net9.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
+    vsTestPlatform: '${{ parameters.vsTestPlatform }}'
+    osName: '${{ parameters.osName }}'
+    testResultsFormat: '${{ parameters.testResultsFormat }}'
+    testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
+    testResultsFileName: '${{ parameters.testResultsFileName }}'
+
+- template: publish-test-results.yml
+  parameters:
+    testProjectName: 'Lucene.Net.Tests.Cli'
+    framework: 'net8.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
     vsTestPlatform: '${{ parameters.vsTestPlatform }}'
     osName: '${{ parameters.osName }}'
     testResultsFormat: '${{ parameters.testResultsFormat }}'
     testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
     testResultsFileName: '${{ parameters.testResultsFileName }}'
 
-# Special case: Only supports net8.0 and net48
+# Special case: Only supports net10.0, net9.0, net8.0 and net48
 
 - template: publish-test-results.yml
   parameters:
     testProjectName: 'Lucene.Net.Tests.Analysis.OpenNLP'
-    framework: 'net8.0' # Since condtions are not supported for templates, we check for the file existence within publish-test-results.yml
+    framework: 'net10.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
+    vsTestPlatform: '${{ parameters.vsTestPlatform }}'
+    osName: '${{ parameters.osName }}'
+    testResultsFormat: '${{ parameters.testResultsFormat }}'
+    testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
+    testResultsFileName: '${{ parameters.testResultsFileName }}'
+
+- template: publish-test-results.yml
+  parameters:
+    testProjectName: 'Lucene.Net.Tests.Analysis.OpenNLP'
+    framework: 'net9.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
+    vsTestPlatform: '${{ parameters.vsTestPlatform }}'
+    osName: '${{ parameters.osName }}'
+    testResultsFormat: '${{ parameters.testResultsFormat }}'
+    testResultsArtifactName: '${{ parameters.testResultsArtifactName }}'
+    testResultsFileName: '${{ parameters.testResultsFileName }}'
+
+- template: publish-test-results.yml
+  parameters:
+    testProjectName: 'Lucene.Net.Tests.Analysis.OpenNLP'
+    framework: 'net8.0' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
     vsTestPlatform: '${{ parameters.vsTestPlatform }}'
     osName: '${{ parameters.osName }}'
     testResultsFormat: '${{ parameters.testResultsFormat }}'
@@ -107,7 +148,7 @@ steps:
 - template: publish-test-results.yml
   parameters:
     testProjectName: 'Lucene.Net.Tests.Analysis.OpenNLP'
-    framework: 'net48' # Since condtions are not supported for templates, we check for the file existence within publish-test-results.yml
+    framework: 'net48' # Since conditions are not supported for templates, we check for the file existence within publish-test-results.yml
     vsTestPlatform: '${{ parameters.vsTestPlatform }}'
     osName: '${{ parameters.osName }}'
     testResultsFormat: '${{ parameters.testResultsFormat }}'

.build/azure-templates/run-tests-on-os.yml

@@ -105,7 +105,7 @@ steps:
     & $installScriptPath -Version $sdkVersion -Architecture $architecture -InstallDir $installPath
     Write-Host "##vso[task.setvariable variable=DOTNET_ROOT_X86;]$installPath"
   displayName: 'Use .NET SDK ${{ parameters.dotNetSdkVersion }} (x86)'
-  condition: and(succeeded(), contains('${{ parameters.framework }}', 'net9.'), eq('${{ parameters.vsTestPlatform }}', 'x86'))
+  condition: and(succeeded(), contains('${{ parameters.framework }}', 'net10.'), eq('${{ parameters.vsTestPlatform }}', 'x86'))
 
 - task: UseDotNet@2
   displayName: 'Use .NET SDK 8.0.404'
@@ -132,6 +132,31 @@ steps:
   displayName: 'Use .NET SDK 8.0.404 (x86)'
   condition: and(succeeded(), contains('${{ parameters.framework }}', 'net8.'), eq('${{ parameters.vsTestPlatform }}', 'x86'))
 
+- task: UseDotNet@2
+  displayName: 'Use .NET SDK 9.0.308'
+  inputs:
+    packageType: 'sdk'
+    version: '9.0.308'
+    performMultiLevelLookup: '${{ variables.PerformMultiLevelLookup }}'
+  condition: and(succeeded(), contains('${{ parameters.framework }}', 'net9.'))
+
+    # Hack: .NET 8+ no longer installs the x86 bits and they must be installed separately. However, it is not
+    # trivial to get it into the path and to get it to pass the minimum SDK version check in runbuild.ps1.
+    # So, we install it afterward and set the environment variable so the above SDK can delegate to it.
+    # This code only works on Windows.
+- pwsh: |
+    $sdkVersion = '9.0.308'
+    $architecture = '${{ parameters.vsTestPlatform }}'
+    $installScriptPath = "${env:AGENT_TEMPDIRECTORY}/dotnet-install.ps1"
+    $installScriptUrl = "https://raw.githubusercontent.com/dotnet/install-scripts/main/src/dotnet-install.ps1"
+    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
+    Invoke-WebRequest $installScriptUrl -OutFile $installScriptPath -TimeoutSec 60
+    $installPath = "${env:ProgramFiles(x86)}/dotnet"
+    & $installScriptPath -Version $sdkVersion -Architecture $architecture -InstallDir $installPath
+    Write-Host "##vso[task.setvariable variable=DOTNET_ROOT_X86;]$installPath"
+  displayName: 'Use .NET SDK 9.0.308 (x86)'
+  condition: and(succeeded(), contains('${{ parameters.framework }}', 'net9.'), eq('${{ parameters.vsTestPlatform }}', 'x86'))
+
 
 #- template: 'show-all-files.yml' # Uncomment for debugging
 - pwsh: |

.build/dependencies.props

@@ -22,7 +22,7 @@
   <PropertyGroup Label="NuGet Package Reference Versions">
     <Antlr4CodeGeneratorPackageVersion>4.6.6</Antlr4CodeGeneratorPackageVersion>
     <Antlr4RuntimeStandardPackageVersion>4.13.1</Antlr4RuntimeStandardPackageVersion>
-    <Antlr4BuildTasksPackageVersion>12.8.0</Antlr4BuildTasksPackageVersion>
+    <Antlr4BuildTasksPackageVersion>12.11.0</Antlr4BuildTasksPackageVersion>
     <!-- LUCENENET TODO: When ICU4N is released to production,
         be sure to lock down the version range below. The resource
         files in Lucene.Net.ICU are not compatible with any other

.build/runbuild.ps1

@@ -270,7 +270,7 @@ task Test -depends CheckSDK, UpdateLocalSDKVersion, Restore -description "This t
         $testName = $testProject.Directory.Name
 
         # Call the target to get the configured test frameworks for this project. We only read the first line because MSBuild adds extra output.
-        $frameworksString = $(dotnet build "$testProject" --verbosity minimal --nologo --no-restore /t:PrintTargetFrameworks /p:TestProjectsOnly=true /p:TestFrameworks=true)[0].Trim()
+        $frameworksString = Get-SupportedTargetFrameworksString $testProject
 
         Write-Host ""
         Write-Host "Frameworks To Test for ${testProject}: $frameworksString" -ForegroundColor Yellow
@@ -417,12 +417,21 @@ function Get-Version() {
 
 function Get-FrameworksToTest() {
     # Call the target to get the configured test frameworks for a project known to contain all of them. We only read the first line because MSBuild adds extra output.
-    $frameworksString = $(dotnet build "$projectWithAllTestFrameworks" --verbosity minimal --nologo --no-restore /t:PrintTargetFrameworks /p:TestProjectsOnly=true /p:TestFrameworks=true)[0].Trim()
+    $frameworksString = Get-SupportedTargetFrameworksString -ProjectPath $projectWithAllTestFrameworks
     $frameworksToTest = $frameworksString -split '\s*;\s*'
     Assert($frameworksToTest.Length -gt 0) "The project at $(Normalize-FileSystemSlashes "$projectWithAllTestFrameworks") contains no target frameworks. Please configure a project that includes all testable target frameworks."
     return $frameworksToTest
 }
 
+function Get-SupportedTargetFrameworksString([Parameter(Mandatory)][string] $ProjectPath) {
+    # NOTE: This will not appear when run directly in the console with minimal verbosity. MSBuild only produces the output when using a pipe, which is what we are doing here.
+    $output = dotnet build "$ProjectPath" --verbosity minimal --nologo --no-restore /t:PrintTargetFrameworks /p:TestProjectsOnly=true /p:TestFrameworks=true 2>&1 | Out-String
+    if ($output -match 'SupportedTargetFrameworks=([^\s]+)') {
+        return $matches[1]
+    }
+    throw "Failed to determine supported target frameworks for project: $ProjectPath"
+}
+
 function Prepare-For-Build() {
     #Use only the major version as the assembly version.
     #This ensures binary compatibility unless the major version changes.

.github/workflows/Generate-TestWorkflows.ps1

@@ -38,7 +38,7 @@
 
  .PARAMETER TestFrameworks
     A string array of Dotnet target framework monikers to run the tests on. The default is
-    @('net9.0','net8.0','net6.0','net472','net48').
+    @('net10.0','net8.0','net472','net48').
 
  .PARAMETER OperatingSystems
     A string array of Github Actions operating system monikers to run the tests on.
@@ -51,9 +51,9 @@
  .PARAMETER Configurations
     A string array of build configurations to run the tests on. The default is @('Release').
 
- .PARAMETER DotNet9SDKVersion
-    The SDK version of .NET 9.x to install on the build agent to be used for building and
-    testing. This SDK is always installed on the build agent. The default is 9.0.x.
+ .PARAMETER DotNet10SDKVersion
+    The SDK version of .NET 10.x to install on the build agent to be used for building and
+    testing. This SDK is always installed on the build agent. The default is 10.0.x.
 
  .PARAMETER DotNet8SDKVersion
     The SDK version of .NET 8.x to install on the build agent to be used for building and
@@ -65,15 +65,15 @@ param(
 
     [string]$RepoRoot = (Split-Path (Split-Path $PSScriptRoot)),
 
-    [string[]]$TestFrameworks = @('net9.0','net8.0','net472','net48'), # targets under test: net8.0, net8.0, netstandard2.0, net462
+    [string[]]$TestFrameworks = @('net10.0', 'net8.0', 'net472', 'net48'), # targets under test: net10.0, net8.0, netstandard2.0, net462
 
     [string[]]$OperatingSystems = @('windows-latest', 'ubuntu-latest'),
 
     [string[]]$TestPlatforms = @('x64'),
 
     [string[]]$Configurations = @('Release'),
 
-    [string]$DotNet9SDKVersion = '9.0.x',
+    [string]$DotNet10SDKVersion = '10.0.x',
 
     [string]$DotNet8SDKVersion = '8.0.x'
 )
@@ -141,6 +141,15 @@ function Get-ProjectPathDirectories([string]$ProjectPath, [string]$RelativeRoot,
     }
 }
 
+function Get-SupportedTargetFrameworksString([Parameter(Mandatory)][string] $ProjectPath) {
+    # NOTE: This will not appear when run directly in the console with minimal verbosity. MSBuild only produces the output when using a pipe, which is what we are doing here.
+    $output = dotnet build "$ProjectPath" --verbosity minimal --nologo --no-restore /t:PrintTargetFrameworks /p:TestProjectsOnly=true /p:TestFrameworks=true 2>&1 | Out-String
+    if ($output -match 'SupportedTargetFrameworks=([^\s]+)') {
+        return $matches[1]
+    }
+    throw "Failed to determine supported target frameworks for project: $ProjectPath"
+}
+
 function Ensure-Directory-Exists([string] $path) {
     if (!(Test-Path $path)) {
         New-Item $path -ItemType Directory
@@ -155,7 +164,7 @@ function Write-TestWorkflow(
     [string[]]$TestFrameworks = @('net6.0', 'net48'),
     [string[]]$TestPlatforms = @('x64'),
     [string[]]$OperatingSystems = @('windows-latest', 'ubuntu-latest', 'macos-latest'),
-    [string]$DotNet9SDKVersion = $DotNet9SDKVersion,
+    [string]$DotNet10SDKVersion = $DotNet10SDKVersion,
     [string]$DotNet8SDKVersion = $DotNet8SDKVersion) {
 
     $dependencies = New-Object System.Collections.Generic.HashSet[string]
@@ -280,10 +289,10 @@ jobs:
           dotnet-version: '$DotNet8SDKVersion'
         if: `${{ startswith(matrix.framework, 'net8.') }}
 
-      - name: Setup .NET 9 SDK
+      - name: Setup .NET 10 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '$DotNet9SDKVersion'
+          dotnet-version: '$DotNet10SDKVersion'
 
       - name: Cache NuGet Packages
         uses: actions/cache@v5
@@ -353,7 +362,10 @@ jobs:
     Ensure-Directory-Exists $OutputDirectory
 
     Write-Host "Generating workflow file: $FilePath"
-    Out-File -filePath $FilePath -encoding UTF8 -inputObject $fileText
+
+    # Ensure the file does not get generated with a BOM
+    $utf8NoBom = New-Object System.Text.UTF8Encoding($false)
+    [System.IO.File]::WriteAllText($FilePath, $fileText, $utf8NoBom)
 
     #Write-Host $fileText
 }
@@ -373,8 +385,8 @@ try {
 foreach ($testProject in $TestProjects) {
     $projectName = [System.IO.Path]::GetFileNameWithoutExtension($testProject)
 
-     # Call the target to get the configured test frameworks for this project. We only read the first line because MSBuild adds extra output.
-    $frameworksString = $(dotnet build "$testProject" --verbosity minimal --nologo --no-restore /t:PrintTargetFrameworks /p:TestProjectsOnly=true /p:TestFrameworks=true)[0].Trim()
+     # Call the target to get the configured test frameworks for this project.
+    $frameworksString = Get-SupportedTargetFrameworksString $testProject
 
     if ($frameworksString -eq 'none') {
         Write-Host "WARNING: Skipping project '$projectName' because it is not marked with `<IsTestProject`>true`<`/IsTestProject`> and/or it contains no test frameworks for the current environment." -ForegroundColor Yellow

.github/workflows/Lucene-Net-Tests-AllProjects.yml

@@ -73,7 +73,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [windows-latest, ubuntu-latest]
-        framework: [net9.0, net8.0, net48, net472]
+        framework: [net10.0, net8.0, net48, net472]
         platform: [x64]
         configuration: [Release]
         exclude:
@@ -102,13 +102,13 @@ jobs:
       - name: Setup .NET 8 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '8.0.415'
+          dotnet-version: '8.0.x'
         if: ${{ startswith(matrix.framework, 'net8.') }}
 
-      - name: Setup .NET 9 SDK
+      - name: Setup .NET 10 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '9.0.306'
+          dotnet-version: '10.0.x'
 
       - name: Cache NuGet Packages
         uses: actions/cache@v5
@@ -158,4 +158,3 @@ jobs:
           if (Test-Path $md_file) {
               Get-Content $md_file | Add-Content $env:GITHUB_STEP_SUMMARY
           }
-

.github/workflows/Lucene-Net-Tests-Analysis-Common.yml

@@ -57,7 +57,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [windows-latest, ubuntu-latest]
-        framework: [net9.0, net8.0, net48, net472]
+        framework: [net10.0, net8.0, net48, net472]
         platform: [x64]
         configuration: [Release]
         exclude:
@@ -86,13 +86,13 @@ jobs:
       - name: Setup .NET 8 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '8.0.415'
+          dotnet-version: '8.0.x'
         if: ${{ startswith(matrix.framework, 'net8.') }}
 
-      - name: Setup .NET 9 SDK
+      - name: Setup .NET 10 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '9.0.306'
+          dotnet-version: '10.0.x'
 
       - name: Cache NuGet Packages
         uses: actions/cache@v5
@@ -142,4 +142,3 @@ jobs:
           if (Test-Path $md_file) {
               Get-Content $md_file | Add-Content $env:GITHUB_STEP_SUMMARY
           }
-

.github/workflows/Lucene-Net-Tests-Analysis-Kuromoji.yml

@@ -54,7 +54,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [windows-latest, ubuntu-latest]
-        framework: [net9.0, net8.0, net48, net472]
+        framework: [net10.0, net8.0, net48, net472]
         platform: [x64]
         configuration: [Release]
         exclude:
@@ -83,13 +83,13 @@ jobs:
       - name: Setup .NET 8 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '8.0.415'
+          dotnet-version: '8.0.x'
         if: ${{ startswith(matrix.framework, 'net8.') }}
 
-      - name: Setup .NET 9 SDK
+      - name: Setup .NET 10 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '9.0.306'
+          dotnet-version: '10.0.x'
 
       - name: Cache NuGet Packages
         uses: actions/cache@v5
@@ -139,4 +139,3 @@ jobs:
           if (Test-Path $md_file) {
               Get-Content $md_file | Add-Content $env:GITHUB_STEP_SUMMARY
           }
-

.github/workflows/Lucene-Net-Tests-Analysis-Morfologik.yml

@@ -54,7 +54,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [windows-latest, ubuntu-latest]
-        framework: [net9.0, net8.0, net48, net472]
+        framework: [net10.0, net8.0, net48, net472]
         platform: [x64]
         configuration: [Release]
         exclude:
@@ -83,13 +83,13 @@ jobs:
       - name: Setup .NET 8 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '8.0.415'
+          dotnet-version: '8.0.x'
         if: ${{ startswith(matrix.framework, 'net8.') }}
 
-      - name: Setup .NET 9 SDK
+      - name: Setup .NET 10 SDK
         uses: actions/setup-dotnet@v5
         with:
-          dotnet-version: '9.0.306'
+          dotnet-version: '10.0.x'
 
       - name: Cache NuGet Packages
         uses: actions/cache@v5
@@ -139,4 +139,3 @@ jobs:
           if (Test-Path $md_file) {
               Get-Content $md_file | Add-Content $env:GITHUB_STEP_SUMMARY
           }
-