Open
Description
The System.Runtime and System.Runtime.Numerics work items consistently fail on the linux outerloop pipeline:
/root/helix/work/workitem/e /root/helix/work/workitem/e
Discovering: System.Runtime.Numerics.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Runtime.Numerics.Tests (found 7 of 562 test cases)
Starting: System.Runtime.Numerics.Tests (parallel test collections = on, max threads = 2)
./RunTests.sh: line 168: 22 Killed "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Runtime.Numerics.Tests.runtimeconfig.json --depsfile System.Runtime.Numerics.Tests.deps.json xunit.console.dll System.Runtime.Numerics.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Fri 07 Jul 2023 11:41:50 AM UTC ----- exit code 137 ----------------------------------------------------------
exit code 137 means SIGKILL Killed eg by kill
Looking further down in the log we can see that they've been killed by the Out of Memory (OOM) killer:
[ 2661.445879] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/docker/6a407bb075622c6a7f6a9a7b88ddd30217d8294911afde258ca18d342721bf3b,task=dotnet,pid=25681,uid=1000
[ 2661.445924] Out of memory: Killed process 25681 (dotnet) total-vm:273925828kB, anon-rss:7218652kB, file-rss:0kB, shmem-rss:14728kB, UID:1000 pgtables:14564kB oom_score_adj:0
This has been going on for as long as there are logs in AzDO (June 19th).