Collecting some build data for tasks/targets telemetry

[JanKrivanek]

Fixes #10946

If split into separate PRs is prefered - please indicate so in comments

Goal

Obtaining information about build composition from microsoft versus 3rd party tasks and targets.

Approach

Since the information is present in the building worker nodes - the code collects it there and then (if requested) send it via WorkerNodeTelemetryEventArgs to the main node.

The classification of 3rd party versus 1st party is for simplicity being done based on location of defining msbuild project and naming of the assembly.

Changes

• Added class for the exexution statistics of tasks - this is contained in TaskRegistry as well as in TaskFactoryWrappers
• RequestBuilder is the orchestration here, that decides whether statistics are needed and if yes - traverses the TaskRegistry, BuildResult and ProjectCollection in order to accumulate and populate the statistics
• Added WorkerNodeTelemetryEventArgs that holds data, InternalTelemeteryConsumingLogger and InternalTelemeteryForwardingLogger to transfer the data (only if telemetry is sampled in)

Performance considerations

By default the data collection logic is off and hence statistics are not collected on worker node, nor serialized to the event args. The perf impact of collection and serialization was though beyond the recognition level of basic 'full duration' testing of full and incremental build of small console and bigger size projects.

Sample of collected data

This is stringified flushed version of stats collected and aggregated from nodes.
Once inputing into our telemetry client, we might send just subset (topN).
We collect the info about whether the task/target is 1st or 3rd party ('C:' prefix), whether it's comming from nuget cache ('N:' prefix) or is generated within metaproj, the task/target name (will be hashed for 3rd party), the duration of the task, the memory consumption of the task and the number of executions of the task

==========================================
Tasks: (258)
Custom tasks:
C:ReplaceFileText
==========================================
Tasks by time:
Microsoft.Build.Tasks.ResolveAssemblyReference - 00:00:34.5173355
Microsoft.CodeAnalysis.BuildTasks.CopyRefAssembly - 00:00:20.0824767
Microsoft.Build.Tasks.WriteLinesToFile - 00:00:08.7755037
Microsoft.NET.Build.Tasks.ResolvePackageAssets - 00:00:07.6207085
Microsoft.CodeAnalysis.BuildTasks.Csc - 00:00:06.6906938
(...)
==========================================
Tasks by memory consumption:
Microsoft.Build.Tasks.ResolveAssemblyReference - 10751256.03 kB
Microsoft.Build.Tasks.AssignProjectConfiguration - 1189559.04 kB
Microsoft.Build.Tasks.AssignTargetPath - 393482.46 kB
Microsoft.Build.Tasks.SetRidAgnosticValueForProjects - 294210.73 kB
(...)
N:Microsoft.Build.Tasks.Git.GetUntrackedFiles - 3543.50 kB
(...)
==========================================
Tasks by Executions count:
GetPackageDirectory - 4140
Microsoft.Build.Tasks.AssignTargetPath - 3050
Microsoft.Build.Tasks.FindUnderPath - 2126
(...)
N:Microsoft.SourceLink.GitHub.GetSourceLinkUrl - 1242
Microsoft.Build.Tasks.RemoveDuplicates - 1086
(...)
=========================================
Targets (1621) - name : executed:
AfterSdkPublish : False
C:MakeWebRootFolder : False
(...)

Performance impact considerations

Somethings always costs more than nothing :-) Beyond that - those changes should be with hardly observable impact.

If telemetry is opted out - there is no extra data collection, processing and sending applied.
When telemetry is opted in - we keep and update hashtables with stats for all tasks and targets in all nodes and at the end of build send those stats to the main node. This minimizes costly traffic during the build.

When measured with 'wall clock time' on incremental build of OrchardCore via:

pskill msbuild; pskill dotnet; pskill vbcscompiler
Measure-Command { & "C:\src\msbuild-2\artifacts\bin\bootstrap\net472\MSBuild\Current\Bin\MSBuild.exe" -v:m | Out-Default }

And excluding warmup and slowest and fastest runs - the versions seems indistinguishable:

main (@aff5455):

Minutes : 2
Seconds : 34
Milliseconds : 352

Minutes : 2
Seconds : 30
Milliseconds : 665

Minutes : 2
Seconds : 30
Milliseconds : 109

This PR (with _isTelemetryEnabled forced to true and $env:MSBUILDOUTPUTNODESTELEMETRY=1):

Minutes : 3
Seconds : 36
Milliseconds : 707

Minutes : 2
Seconds : 37
Milliseconds : 763

Minutes : 2
Seconds : 35
Milliseconds : 794

[Copilot]

Copilot reviewed 19 out of 34 changed files in this pull request and generated 5 comments.

Files not reviewed (15)

• src/Build/Microsoft.Build.csproj: Language not supported
• src/Build.UnitTests/BackEnd/NodePackets_Tests.cs: Evaluated as low risk
• src/Build/BackEnd/Components/BuildComponentFactoryCollection.cs: Evaluated as low risk
• src/Build/Definition/Toolset.cs: Evaluated as low risk
• src/Build/BackEnd/Components/RequestBuilder/TaskBuilder.cs: Evaluated as low risk
• src/Build/BackEnd/Components/Logging/CentralForwardingLogger.cs: Evaluated as low risk
• src/Build/BackEnd/Components/RequestBuilder/TargetBuilder.cs: Evaluated as low risk
• src/Build/BackEnd/Components/IBuildComponentHost.cs: Evaluated as low risk
• src/Framework.UnitTests/FileClassifier_Tests.cs: Evaluated as low risk
• src/Build/BackEnd/BuildManager/BuildParameters.cs: Evaluated as low risk
• src/Build/BackEnd/Components/BuildRequestEngine/BuildRequestEngine.cs: Evaluated as low risk
• src/Build/BackEnd/Shared/TargetResult.cs: Evaluated as low risk
• src/Build/BackEnd/Components/Logging/EventSourceSink.cs: Evaluated as low risk
• src/Build/Instance/TaskFactoryWrapper.cs: Evaluated as low risk
• src/Build/BackEnd/BuildManager/BuildManager.cs: Evaluated as low risk

Comments suppressed due to low confidence (2)

src/Build/BackEnd/Components/RequestBuilder/RequestBuilder.cs:1263

• The new telemetry collection logic in 'UpdateStatisticsPostBuild' is not covered by tests. Add test cases to ensure this logic is tested.

private void UpdateStatisticsPostBuild()

src/Build/Instance/TaskRegistry.cs:1182

• The 'Stats' class should be instantiated with 'new Stats()' instead of 'new Stats'.

internal class Stats()

[surayya-MS]

On the first round of review it looks good! I'll look more into this

[surayya-MS]

Looks great!

[JanProvaznik]

I left a few suggestions if you agree with the naming. All in all: