UnixDomainSocket test is timing out

[rmkerr]

The recently added test ConcurrentSendReceive has failed in official runs with the following error 1-2 times a day for the last three days:

System.TimeoutException : WhenAllOrAnyFailed timed out after 10000
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
   at System.Net.Sockets.Tests.UnixDomainSocketTest.ConcurrentSendReceive(Boolean forceNonBlocking) in E:\A\_work\2\s\corefx\src\System.Net.Sockets\tests\FunctionalTests\UnixDomainSocketTest.netcoreapp.cs:line 284
--- End of stack trace from previous location where exception was thrown ---

The failures occur for both the blocking and non-blocking case, and has occurred exclusively on the Windows.10.Amd64.ClientRS4 leg.

I'm not sure if that means Unix domain sockets are not enabled in RS4, if there is some config that needs to happen on that machine, or if there is a real bug here.

[stephentoub]

@rmkerr, you said it's failing 1-2 times a day, but it's not clear to me: is that all of the RS4 runs, or are there runs where it succeeds and it's just failing sporadically?

I'm not sure if that means Unix domain sockets are not enabled in RS4

RS4 is the first released version of Windows where they are supported.

[rmkerr]

It's failing sporadically. The test has been run 24 times over the last four days, with six failures. It doesn't look like a configuration issue, since sometimes only one of the two cases will fail.

[stephentoub]

And it's just that one test (ConcurrentSendReceive), right? The others are all fine?

[stephentoub]

And you mentioned "official runs"... what about PRs? It's an inner loop test, so it should be running now as part of every PR. Has it failed there?

[rmkerr]

Looking into it now. It looks like ConcurrentSendRecieveAsync has one failure on the 15th. I'll check PRs as well.

[stephentoub]

@sunilmut, are there any known issues in RS4 with having multiple threads issuing sends/recvs on the same unix domain socket at the same time?

[rmkerr]

It looks like we're only running RS3 in PRs. So this test is always being skipped on PRs, and is only enabled on the official runs.

[rmkerr]

@stephentoub I talked to Sunil about this -- there are not any known issues around multiple sends/receives. I'll try to see if I can reproduce this locally.

[stephentoub]

@rmkerr, is this still happening?

[rmkerr]

My setup for checking in on the tests was just broken by the move away from ci.dot.net, so I'm not sure. I'll try to get it running again today though.

[rmkerr]

Based on this recent test run, it looks like this is still happening.

[wfurt]

failed 20x in last 30 days in official runs. all in Windows.10.Amd64.ClientRS4 queue.

   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in F:\vsagent\11\s\src\Common\tests\System\Threading\Tasks\TaskTimeoutExtensions.cs:line 69
   at System.Net.Sockets.Tests.UnixDomainSocketTest.ConcurrentSendReceiveAsync() in F:\vsagent\11\s\src\System.Net.Sockets\tests\FunctionalTests\UnixDomainSocketTest.netcoreapp.cs:line 322
--- End of stack trace from previous location where exception was thrown ---

[wfurt]

This seems specific to RS4. Another 26 failures in last 20 days.

Should we disable this test on that OS?
Perhaps push it to Feature?
Any recommendation for instrumentation?

cc: @davidsh

[stephentoub]

This seems specific to RS4.

Is it running and passing on RS5?

[davidsh]

Should we disable this test on that OS?
Perhaps push it to Feature?
Any recommendation for instrumentation?
cc: @davidsh

This is another case of asking ourselves are we adding value by testing this in CI with our .NET layer. I don't know which version of Windows 10 the UnixDomainSocket became operational. Maybe there are some bugs in it. We would have to ask the Windows TCP team.

Unless we want to spend a lot of time investigating a Windows native feature, we should either disable this test on Windows or at least restrict it to RS5 and above. That would give us some test coverage on Windows and reduce the noise in our test results.