Intermittent failure when downloading artifacts #1112
Description
We use the OneOffInsertion.ps1 script to do our insertion into VS process. We have been seeing and issue with it where it fails to download the artifacts. Here are some recent runs with this failure:
- https://devdiv.visualstudio.com/DevDiv/_releaseProgress?releaseId=1193089&_a=release-environment-logs&environmentId=6072101
- https://devdiv.visualstudio.com/DevDiv/_releaseProgress?releaseId=1192212&environmentId=6067436&_a=release-environment-logs
- https://devdiv.visualstudio.com/DevDiv/_releaseProgress?releaseId=1186001&environmentId=6032876&_a=release-environment-logs
- https://devdiv.visualstudio.com/DevDiv/_releaseProgress?releaseId=1184103&environmentId=6022016&_a=release-environment-logs
- https://devdiv.visualstudio.com/DevDiv/_releaseProgress?releaseId=1186001&environmentId=6032876&_a=release-environment-logs
They usually appear in the Attempt 1 of deployment. We manually rerun the deployment when it fails:
Here is the callstack from one of the failures:
2021-10-14T13:01:27.1925369Z Downloading artifacts to C:\Users\AzDevOps\AppData\Local\Temp\1\Project_Systemmain\20211014.1
2021-10-14T13:03:28.0521799Z ##[error] System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
2021-10-14T13:03:28.0534447Z ##[debug]Processed: ##vso[task.logissue type=error] System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
2021-10-14T13:03:28.0536665Z at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
2021-10-14T13:03:28.0537389Z at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2021-10-14T13:03:28.0537894Z --- End of inner exception stack trace ---
2021-10-14T13:03:28.0538324Z at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
2021-10-14T13:03:28.0539585Z at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
2021-10-14T13:03:28.0541512Z --- End of stack trace from previous location where exception was thrown ---
2021-10-14T13:03:28.0542083Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-10-14T13:03:28.0542655Z at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-10-14T13:03:28.0543283Z at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.<ReadAsync>d__4.MoveNext()
2021-10-14T13:03:28.0543848Z --- End of stack trace from previous location where exception was thrown ---
2021-10-14T13:03:28.0544371Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-10-14T13:03:28.0545026Z at System.IO.Stream.<CopyToAsyncInternal>d__27.MoveNext()
2021-10-14T13:03:28.0545526Z --- End of stack trace from previous location where exception was thrown ---
2021-10-14T13:03:28.0546048Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-10-14T13:03:28.0546661Z at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-10-14T13:03:28.0547284Z at Roslyn.Insertion.RoslynInsertionTool.<DownloadBuildArtifactsAsync>d__51.MoveNext()
2021-10-14T13:03:28.0547850Z --- End of stack trace from previous location where exception was thrown ---
2021-10-14T13:03:28.0548384Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-10-14T13:03:28.0548926Z at Roslyn.Insertion.RoslynInsertionTool.<GetInsertionArtifactsAsync>d__50.MoveNext()
2021-10-14T13:03:28.0549502Z --- End of stack trace from previous location where exception was thrown ---
2021-10-14T13:03:28.0550129Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-10-14T13:03:28.0550704Z at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-10-14T13:03:28.0551304Z at Roslyn.Insertion.RoslynInsertionTool.<PerformInsertionAsync>d__13.MoveNext()
@JoeRobich @RikkiGibson The failure seems to stem from here: