Skip to content

[Profiler] Investigation alpine deadlock#8005

Closed
gleocadie wants to merge 1 commit intomasterfrom
gleocadie/investigate-deadlock-alpine
Closed

[Profiler] Investigation alpine deadlock#8005
gleocadie wants to merge 1 commit intomasterfrom
gleocadie/investigate-deadlock-alpine

Conversation

@gleocadie
Copy link
Copy Markdown
Collaborator

@gleocadie gleocadie commented Dec 23, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@github-actions github-actions bot added the area:profiler Issues related to the continous-profiler label Dec 23, 2025
@dd-trace-dotnet-ci-bot
Copy link
Copy Markdown

dd-trace-dotnet-ci-bot bot commented Dec 23, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (8005) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration76.17 ± (76.23 - 76.65) ms76.67 ± (76.62 - 76.97) ms+0.7%✅⬆️
.NET Framework 4.8 - Bailout
duration80.36 ± (80.25 - 80.61) ms80.86 ± (80.71 - 81.04) ms+0.6%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1075.22 ± (1082.57 - 1094.21) ms1078.97 ± (1083.10 - 1095.00) ms+0.3%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms23.06 ± (23.01 - 23.10) ms23.31 ± (23.26 - 23.36) ms+1.1%✅⬆️
process.time_to_main_ms87.57 ± (87.35 - 87.79) ms87.61 ± (87.41 - 87.81) ms+0.0%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.92 ± (10.92 - 10.93) MB10.92 ± (10.91 - 10.92) MB-0.1%✅
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms23.05 ± (22.99 - 23.10) ms23.22 ± (23.17 - 23.27) ms+0.8%✅⬆️
process.time_to_main_ms88.64 ± (88.47 - 88.81) ms89.33 ± (89.11 - 89.56) ms+0.8%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.96 ± (10.95 - 10.96) MB10.96 ± (10.95 - 10.96) MB-0.0%✅
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms261.69 ± (257.42 - 265.95) ms265.05 ± (261.00 - 269.10) ms+1.3%✅⬆️
process.time_to_main_ms507.50 ± (506.62 - 508.38) ms508.13 ± (507.23 - 509.02) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.16 ± (48.14 - 48.18) MB48.15 ± (48.14 - 48.17) MB-0.0%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.2%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms21.87 ± (21.83 - 21.91) ms21.80 ± (21.76 - 21.84) ms-0.3%✅
process.time_to_main_ms76.04 ± (75.86 - 76.22) ms75.69 ± (75.53 - 75.85) ms-0.5%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.62 ± (10.62 - 10.63) MB10.63 ± (10.63 - 10.64) MB+0.1%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.84 ± (21.78 - 21.90) ms21.64 ± (21.59 - 21.68) ms-0.9%✅
process.time_to_main_ms77.65 ± (77.46 - 77.84) ms76.46 ± (76.29 - 76.63) ms-1.5%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.72 ± (10.71 - 10.73) MB10.74 ± (10.74 - 10.74) MB+0.2%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms258.53 ± (255.86 - 261.20) ms250.06 ± (246.17 - 253.96) ms-3.3%✅
process.time_to_main_ms470.46 ± (469.79 - 471.13) ms471.62 ± (470.92 - 472.33) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.52 ± (48.50 - 48.55) MB48.47 ± (48.44 - 48.49) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.1%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms20.23 ± (20.18 - 20.28) ms20.04 ± (19.99 - 20.08) ms-1.0%✅
process.time_to_main_ms75.84 ± (75.66 - 76.01) ms75.17 ± (75.01 - 75.33) ms-0.9%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.67 ± (7.67 - 7.68) MB7.68 ± (7.67 - 7.69) MB+0.1%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms20.21 ± (20.15 - 20.27) ms20.08 ± (20.04 - 20.13) ms-0.6%✅
process.time_to_main_ms77.03 ± (76.84 - 77.21) ms76.25 ± (76.06 - 76.44) ms-1.0%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.73 ± (7.72 - 7.74) MB7.72 ± (7.71 - 7.73) MB-0.1%✅
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms192.68 ± (191.94 - 193.42) ms192.47 ± (191.76 - 193.19) ms-0.1%✅
process.time_to_main_ms455.90 ± (455.28 - 456.51) ms456.50 ± (455.92 - 457.07) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed36.57 ± (36.53 - 36.61) MB36.61 ± (36.57 - 36.64) MB+0.1%✅⬆️
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)-0.5%✅

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration196.21 ± (195.69 - 196.64) ms193.85 ± (194.17 - 195.05) ms-1.2%✅
.NET Framework 4.8 - Bailout
duration197.55 ± (197.26 - 198.01) ms197.40 ± (197.31 - 197.67) ms-0.1%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1109.77 ± (1112.76 - 1120.89) ms1107.66 ± (1109.97 - 1117.27) ms-0.2%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms188.27 ± (187.91 - 188.63) ms187.65 ± (187.37 - 187.93) ms-0.3%✅
process.time_to_main_ms80.62 ± (80.41 - 80.84) ms80.74 ± (80.54 - 80.94) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.05 ± (16.02 - 16.08) MB16.16 ± (16.13 - 16.19) MB+0.7%✅⬆️
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (20 - 20)+0.1%✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms187.40 ± (187.05 - 187.76) ms187.65 ± (187.34 - 187.96) ms+0.1%✅⬆️
process.time_to_main_ms81.95 ± (81.81 - 82.09) ms82.24 ± (82.08 - 82.39) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.14 ± (16.11 - 16.16) MB16.23 ± (16.20 - 16.26) MB+0.6%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (21 - 21)-0.2%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms425.28 ± (421.90 - 428.66) ms415.11 ± (411.62 - 418.59) ms-2.4%✅
process.time_to_main_ms476.05 ± (475.19 - 476.90) ms473.69 ± (473.04 - 474.34) ms-0.5%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed58.71 ± (58.59 - 58.82) MB58.82 ± (58.70 - 58.95) MB+0.2%✅⬆️
runtime.dotnet.threads.count30 ± (29 - 30)29 ± (29 - 30)-0.1%✅
.NET 6 - Baseline
process.internal_duration_ms196.30 ± (195.79 - 196.81) ms192.30 ± (191.89 - 192.72) ms-2.0%✅
process.time_to_main_ms71.01 ± (70.78 - 71.23) ms69.89 ± (69.71 - 70.07) ms-1.6%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.34 ± (16.32 - 16.36) MB16.35 ± (16.27 - 16.43) MB+0.1%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)18 ± (18 - 19)-3.5%✅
.NET 6 - Bailout
process.internal_duration_ms192.64 ± (192.20 - 193.08) ms192.04 ± (191.62 - 192.47) ms-0.3%✅
process.time_to_main_ms71.08 ± (70.92 - 71.25) ms71.03 ± (70.92 - 71.14) ms-0.1%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.19 ± (16.07 - 16.31) MB15.83 ± (15.66 - 15.99) MB-2.3%✅
runtime.dotnet.threads.count19 ± (19 - 20)19 ± (19 - 19)-2.3%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms456.26 ± (453.79 - 458.73) ms455.01 ± (452.98 - 457.03) ms-0.3%✅
process.time_to_main_ms448.99 ± (447.89 - 450.08) ms445.83 ± (445.28 - 446.38) ms-0.7%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed58.29 ± (58.16 - 58.42) MB58.38 ± (58.26 - 58.50) MB+0.2%✅⬆️
runtime.dotnet.threads.count30 ± (29 - 30)29 ± (29 - 30)-0.1%✅
.NET 8 - Baseline
process.internal_duration_ms188.99 ± (188.63 - 189.34) ms190.37 ± (189.98 - 190.75) ms+0.7%✅⬆️
process.time_to_main_ms69.31 ± (69.10 - 69.52) ms69.37 ± (69.20 - 69.55) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.75 ± (11.68 - 11.81) MB11.72 ± (11.69 - 11.74) MB-0.3%✅
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)+0.5%✅⬆️
.NET 8 - Bailout
process.internal_duration_ms188.11 ± (187.90 - 188.33) ms189.15 ± (188.83 - 189.47) ms+0.5%✅⬆️
process.time_to_main_ms70.00 ± (69.91 - 70.08) ms70.37 ± (70.27 - 70.48) ms+0.5%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.83 ± (11.77 - 11.88) MB11.69 ± (11.60 - 11.79) MB-1.1%✅
runtime.dotnet.threads.count19 ± (18 - 19)19 ± (18 - 19)-0.1%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms363.95 ± (362.31 - 365.60) ms365.85 ± (364.17 - 367.53) ms+0.5%✅⬆️
process.time_to_main_ms430.98 ± (430.20 - 431.75) ms429.68 ± (429.11 - 430.24) ms-0.3%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed47.94 ± (47.90 - 47.97) MB47.96 ± (47.93 - 47.99) MB+0.0%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)+0.1%✅⬆️
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (77ms)  : 74, 79
    master - mean (76ms)  : 73, 80

    section Bailout
    This PR (8005) - mean (81ms)  : 79, 83
    master - mean (80ms)  : 79, 82

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (1,089ms)  : 1000, 1178
    master - mean (1,088ms)  : 1001, 1175
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (118ms)  : 115, 121
    master - mean (118ms)  : 114, 122

    section Bailout
    This PR (8005) - mean (120ms)  : 116, 123
    master - mean (119ms)  : 117, 121

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (804ms)  : 743, 866
    master - mean (809ms)  : 747, 872
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (104ms)  : 101, 107
    master - mean (105ms)  : 102, 107

    section Bailout
    This PR (8005) - mean (105ms)  : 102, 107
    master - mean (106ms)  : 104, 108

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (753ms)  : 687, 820
    master - mean (765ms)  : 714, 816
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (103ms)  : 100, 106
    master - mean (104ms)  : 101, 108

    section Bailout
    This PR (8005) - mean (104ms)  : 102, 107
    master - mean (105ms)  : 102, 108

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (680ms)  : 658, 702
    master - mean (685ms)  : 662, 709
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (195ms)  : 189, 201
    master - mean (196ms)  : 190, 203

    section Bailout
    This PR (8005) - mean (197ms)  : 196, 199
    master - mean (198ms)  : 194, 201

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (1,114ms)  : 1060, 1167
    master - mean (1,117ms)  : 1058, 1176
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (276ms)  : 272, 281
    master - mean (278ms)  : 271, 284

    section Bailout
    This PR (8005) - mean (278ms)  : 273, 283
    master - mean (278ms)  : 273, 283

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (918ms)  : 859, 976
    master - mean (930ms)  : 879, 981
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (270ms)  : 265, 275
    master - mean (276ms)  : 266, 286

    section Bailout
    This PR (8005) - mean (271ms)  : 266, 276
    master - mean (273ms)  : 265, 281

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (929ms)  : 903, 956
    master - mean (933ms)  : 886, 980
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8005) - mean (269ms)  : 263, 275
    master - mean (268ms)  : 262, 274

    section Bailout
    This PR (8005) - mean (269ms)  : 264, 274
    master - mean (267ms)  : 264, 271

    section CallTarget+Inlining+NGEN
    This PR (8005) - mean (826ms)  : 808, 844
    master - mean (825ms)  : 803, 847
Loading

@gleocadie gleocadie force-pushed the gleocadie/investigate-deadlock-alpine branch from ac9e418 to 86a7237 Compare December 29, 2025 12:44
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Dec 29, 2025 •
edited
Loading

Benchmarks

Benchmark execution time: 2025-12-29 17:17:45

Comparing candidate commit 21250f9 in PR branch gleocadie/investigate-deadlock-alpine with baseline commit 4fb09d3 in branch master.

Found 9 performance improvements and 8 performance regressions! Performance is the same for 154 metrics, 15 unstable metrics.

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

  • 🟥 execution_time [+18.264ms; +24.641ms] or [+9.342%; +12.604%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net6.0

  • 🟩 execution_time [-16.766ms; -12.478ms] or [-7.948%; -5.915%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net472

  • 🟥 execution_time [+15.903ms; +20.812ms] or [+7.950%; +10.404%]
  • 🟥 throughput [-248992.836op/s; -177051.697op/s] or [-7.700%; -5.475%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark netcoreapp3.1

  • 🟥 execution_time [+30.804µs; +63.381µs] or [+6.917%; +14.233%]
  • 🟥 throughput [-235.077op/s; -122.443op/s] or [-10.461%; -5.449%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472

  • 🟩 throughput [+73.954op/s; +100.544op/s] or [+7.365%; +10.014%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net472

  • 🟩 execution_time [-73.602µs; -68.238µs] or [-6.046%; -5.605%]
  • 🟩 throughput [+48.816op/s; +52.835op/s] or [+5.943%; +6.432%]

scenario:Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery net6.0

  • 🟥 execution_time [+10.048ms; +13.306ms] or [+5.062%; +6.704%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net6.0

  • 🟩 execution_time [-19.946ms; -15.936ms] or [-9.341%; -7.463%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark netcoreapp3.1

  • 🟩 execution_time [-7.459µs; -3.595µs] or [-13.338%; -6.427%]
  • 🟩 throughput [+1067.240op/s; +2290.561op/s] or [+5.858%; +12.572%]

scenario:Benchmarks.Trace.NLogBenchmark.EnrichedLog net6.0

  • 🟩 throughput [+15250.302op/s; +17402.228op/s] or [+8.222%; +9.382%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

  • 🟥 execution_time [+17.370ms; +22.087ms] or [+8.747%; +11.122%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1

  • 🟩 execution_time [-17.994ms; -12.807ms] or [-8.478%; -6.034%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0

  • 🟥 execution_time [+14.536ms; +20.026ms] or [+7.250%; +9.988%]

@datadog-official

This comment has been minimized.

Sign in to view
@gleocadie gleocadie force-pushed the gleocadie/investigate-deadlock-alpine branch from 86a7237 to 21250f9 Compare December 29, 2025 16:19
@gleocadie gleocadie closed this Dec 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area:profiler Issues related to the continous-profiler

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gleocadie