BasicProcessInfoNoSuspendTest and BasicProcessInfoNoSuspendTestAsync are flaky

[hoyosjs]

[mikelle-rogers]

Adding BasicProcessInfoSuspendTest to the list of flaky tests. Here is the stack trace:

at System.IO.BinaryReader.InternalRead(Int32 numBytes)
at System.IO.BinaryReader.ReadUInt16()
at Microsoft.Diagnostics.NETCore.Client.IpcHeader.Parse(BinaryReader reader) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcHeader.cs:line 56
at Microsoft.Diagnostics.NETCore.Client.IpcMessage.Parse(Stream stream) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcMessage.cs:line 116
at Microsoft.Diagnostics.NETCore.Client.IpcClient.Read(Stream stream) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcClient.cs:line 107
at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessageGetContinuation(IpcEndpoint endpoint, IpcMessage message) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcClient.cs:line 44
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.TryGetProcessInfo2() in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 346
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.GetProcessInfo() in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 318
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClientApiShim.GetProcessInfo(TimeSpan timeout) in //src/tests/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClientApiShim.cs:line 50
at Microsoft.Diagnostics.NETCore.Client.GetProcessInfoTests.BasicProcessInfoTestCore(Boolean useAsync, Boolean suspend) in //src/tests/Microsoft.Diagnostics.NETCore.Client/GetProcessInfoTests.cs:line 62
--- End of stack trace from previous location ---

[mikelle-rogers]

BasicProcessInfoSuspendTestAsync

System.IO.EndOfStreamException : Attempted to read past the end of the stream.

at Microsoft.Diagnostics.NETCore.Client.StreamExtensions.ReadBytesAsync(Stream stream, Int32 length, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/StreamExtensions.cs:line 26
at Microsoft.Diagnostics.NETCore.Client.IpcHeader.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcHeader.cs:line 70
at Microsoft.Diagnostics.NETCore.Client.IpcMessage.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcMessage.cs:line 125
at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessageGetContinuationAsync(IpcEndpoint endpoint, IpcMessage message, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcClient.cs:line 83
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.TryGetProcessInfo2Async(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 353
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.GetProcessInfoAsync(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 332
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClientApiShim.GetProcessInfo(TimeSpan timeout) in //src/tests/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClientApiShim.cs:line 46
at Microsoft.Diagnostics.NETCore.Client.GetProcessInfoTests.BasicProcessInfoTestCore(Boolean useAsync, Boolean suspend) in //src/tests/Microsoft.Diagnostics.NETCore.Client/GetProcessInfoTests.cs:line 62
--- End of stack trace from previous location where exception was thrown ---

[mikelle-rogers]

BasicProcessInfoNoSuspendTestAsync

Error message
System.IO.EndOfStreamException : Attempted to read past the end of the stream.

Stack trace
at Microsoft.Diagnostics.NETCore.Client.StreamExtensions.ReadBytesAsync(Stream stream, Int32 length, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/StreamExtensions.cs:line 23
at Microsoft.Diagnostics.NETCore.Client.IpcHeader.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcHeader.cs:line 70
at Microsoft.Diagnostics.NETCore.Client.IpcMessage.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcMessage.cs:line 125
at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessageGetContinuationAsync(IpcEndpoint endpoint, IpcMessage message, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcClient.cs:line 83
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.TryGetProcessInfo2Async(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 353
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.GetProcessInfoAsync(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 332
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClientApiShim.GetProcessInfo(TimeSpan timeout) in //src/tests/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClientApiShim.cs:line 46
at Microsoft.Diagnostics.NETCore.Client.GetProcessInfoTests.GetProcessInfoWithEntrypointAsync(DiagnosticsClientApiShim shim) in //src/tests/Microsoft.Diagnostics.NETCore.Client/GetProcessInfoTests.cs:line 106
at Microsoft.Diagnostics.NETCore.Client.GetProcessInfoTests.BasicProcessInfoTestCore(Boolean useAsync, Boolean suspend) in /_/src/tests/Microsoft.Diagnostics.NETCore.Client/GetProcessInfoTests.cs:line 71
--- End of stack trace from previous location ---

[mikelle-rogers]

BasicProcessInfoSuspendTestAsync

Error message
System.Threading.Tasks.TaskCanceledException : A task was canceled.

Stack trace
at Microsoft.Diagnostics.NETCore.Client.StreamExtensions.ReadBytesAsync(Stream stream, Int32 length, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/StreamExtensions.cs:line 20
at Microsoft.Diagnostics.NETCore.Client.IpcHeader.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcHeader.cs:line 70
at Microsoft.Diagnostics.NETCore.Client.IpcMessage.ParseAsync(Stream stream, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcMessage.cs:line 125
at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessageGetContinuationAsync(IpcEndpoint endpoint, IpcMessage message, CancellationToken cancellationToken) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcClient.cs:line 83
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.TryGetProcessInfo2Async(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 353
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.GetProcessInfoAsync(CancellationToken token) in //src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClient.cs:line 332
at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClientApiShim.GetProcessInfo(TimeSpan timeout) in //src/tests/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClientApiShim.cs:line 46
at Microsoft.Diagnostics.NETCore.Client.GetProcessInfoTests.BasicProcessInfoTestCore(Boolean useAsync, Boolean suspend) in //src/tests/Microsoft.Diagnostics.NETCore.Client/GetProcessInfoTests.cs:line 62
--- End of stack trace from previous location ---

[mikelle-rogers]

@josalem Do you have a hunch as to what is going on?

[josalem]

These look like two different types of errors:

1. "Read Past End of Stream"
2. TaskCanceledException

Both indicate an issue with the target process. In case 1, I'm betting the target process exited unceremoniously either by force or a crash. The latter implies a timing bug in the test harness. If you are trying to triage them further, you'll want to try and repro them locally. If they don't easily repro locally, then that lends credence to this being a timing issue. You could try limiting the test to 2 cores and then straining the cores while the test is running, e.g., run multiple instances of the test at once in a while true loop. If all else fails, open a PR with some logging in smart locations and hope we can catch it in the act in CI with more information.