Skip to content

Linux Wasm AOT microbenchmarks failing due to single failure in some Perf_File tests #2582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
LoopedBard3 opened this issue Aug 30, 2022 · 1 comment
Closed

Linux Wasm AOT microbenchmarks failing due to single failure in some Perf_File tests #2582

LoopedBard3 opened this issue Aug 30, 2022 · 1 comment
Labels
bug Something isn't working pipeline blocker issue blocks perf pipeline stays green. This be considered high priority.

Comments

@LoopedBard3
Copy link
Member

Linux Wasm AOT microbenchmarks runs are failing due to single failures in some Perf_File tests. The failure appears to be a 'IOException: File Exists' and has been seen with the Perf_File tests ReadAllBytes, CopyTo, and CopyToOverwrite. This failure is causing the pipeline run to come back red, although the rest of the tests are still run and uploaded. An example of this failure is as follows:

[2022/08/30 10:36:09][INFO] // Found 1 benchmarks:
[2022/08/30 10:36:09][INFO] //   Perf_File.ReadAllBytes: Job-UGTRCQ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=1048576]
[2022/08/30 10:36:09][INFO] 
[2022/08/30 10:36:09][INFO] // **************************
[2022/08/30 10:36:09][INFO] // Benchmark: Perf_File.ReadAllBytes: Job-UGTRCQ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=1048576]
[2022/08/30 10:36:09][INFO] // *** Execute ***
[2022/08/30 10:36:09][INFO] // Launch: 1 / 1
[2022/08/30 10:36:09][INFO] // Execute: /home/helixbot/.jsvu/v8 --expose_wasm test-main.js -- --run bed188e1-a25a-49c1-82bf-8a348104c9ed.dll --benchmarkName "System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576)" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 48  in /home/helixbot/work/A87D0968/w/A38108F2/e/performance/artifacts/bin/for-running/MicroBenchmarks/bed188e1-a25a-49c1-82bf-8a348104c9ed/bin/net7.0/browser-wasm/AppBundle
[2022/08/30 10:36:09][INFO] Failed to set up high priority. Make sure you have the right permissions. Message: Permission denied
[2022/08/30 10:36:09][INFO] Incoming arguments: --run bed188e1-a25a-49c1-82bf-8a348104c9ed.dll --benchmarkName System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
[2022/08/30 10:36:09][INFO] Application arguments: --run bed188e1-a25a-49c1-82bf-8a348104c9ed.dll --benchmarkName System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
[2022/08/30 10:36:09][INFO] console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
[2022/08/30 10:36:09][INFO] console.info: Initializing.....
[2022/08/30 10:36:09][INFO] // BeforeAnythingElse
[2022/08/30 10:36:09][INFO] 
[2022/08/30 10:36:09][INFO] // Benchmark Process Environment Information:
[2022/08/30 10:36:09][INFO] // Runtime=.NET Core (Mono) 8.0.0-ci, Wasm NativeAOT
[2022/08/30 10:36:09][INFO] // GC=Non-concurrent Workstation
[2022/08/30 10:36:09][INFO] // HardwareIntrinsics=
[2022/08/30 10:36:09][INFO] // Job: Job-OIZOZY(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
[2022/08/30 10:36:09][INFO] 
[2022/08/30 10:36:09][INFO] 
[2022/08/30 10:36:09][INFO] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
[2022/08/30 10:36:09][INFO]  ---> System.IO.IOException: File exists
[2022/08/30 10:36:09][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
[2022/08/30 10:36:09][INFO]    at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
[2022/08/30 10:36:09][INFO]    --- End of inner exception stack trace ---
[2022/08/30 10:36:09][INFO]    at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
[2022/08/30 10:36:09][INFO] // AfterAll
[2022/08/30 10:36:09][INFO] {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
[2022/08/30 10:36:10][INFO] No Workload Results were obtained from the run.
[2022/08/30 10:36:10][INFO] // Benchmark Process 9672 has exited with code 255.
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // ** Remained 79 (61.7%) benchmark(s) to run. Estimated finish 2022-08-30 10:49 (0h 13m from now) **
[2022/08/30 10:36:10][INFO] // ***** BenchmarkRunner: Finish  *****
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // * Export *
[2022/08/30 10:36:10][INFO]   home/helixbot/work/A87D0968/w/A38108F2/e/artifacts/BenchmarkDotNet.Artifacts/results/System.IO.Tests.Perf_File-report-github.md
[2022/08/30 10:36:10][INFO]   home/helixbot/work/A87D0968/w/A38108F2/e/artifacts/BenchmarkDotNet.Artifacts/results/System.IO.Tests.Perf_File-report-full.json
[2022/08/30 10:36:10][INFO]   home/helixbot/work/A87D0968/w/A38108F2/e/artifacts/BenchmarkDotNet.Artifacts/results/System.IO.Tests.Perf_File-perf-lab-report.json
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // * Detailed results *
[2022/08/30 10:36:10][INFO] Perf_File.ReadAllBytes: Job-UGTRCQ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=1048576]
[2022/08/30 10:36:10][INFO] Runtime = .NET Core (Mono) 8.0.0-ci, Wasm NativeAOT; GC = Non-concurrent Workstation
[2022/08/30 10:36:10][INFO] There are not any results runs
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // * Summary *
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] BenchmarkDotNet=v0.13.1.1847-nightly, OS=ubuntu 18.04
[2022/08/30 10:36:10][INFO] Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
[2022/08/30 10:36:10][INFO] .NET SDK=7.0.100-rc.2.22426.5
[2022/08/30 10:36:10][INFO]   [Host]     : .NET 7.0.0 (7.0.22.42212), X64 RyuJIT AVX2
[2022/08/30 10:36:10][INFO]   Job-UGTRCQ : .NET Core (Mono) 8.0.0-ci, Wasm NativeAOT
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] PowerPlanMode=00000000-0000-0000-0000-000000000000  Runtime=Wasm  Toolchain=Wasm
[2022/08/30 10:36:10][INFO] IterationTime=250.0000 ms  MaxIterationCount=20  MinIterationCount=15
[2022/08/30 10:36:10][INFO] WarmupCount=1
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] |       Method |    size | Mean | Error | Median | Min | Max |
[2022/08/30 10:36:10][INFO] |------------- |-------- |-----:|------:|-------:|----:|----:|
[2022/08/30 10:36:10][INFO] | ReadAllBytes | 1048576 |   NA |    NA |     NA |  NA |  NA |
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] Benchmarks with issues:
[2022/08/30 10:36:10][INFO]   Perf_File.ReadAllBytes: Job-UGTRCQ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=1048576]
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // * Legends *
[2022/08/30 10:36:10][INFO]   size   : Value of the 'size' parameter
[2022/08/30 10:36:10][INFO]   Mean   : Arithmetic mean of all measurements
[2022/08/30 10:36:10][INFO]   Error  : Half of 99.9% confidence interval
[2022/08/30 10:36:10][INFO]   Median : Value separating the higher half of all measurements (50th percentile)
[2022/08/30 10:36:10][INFO]   Min    : Minimum
[2022/08/30 10:36:10][INFO]   Max    : Maximum
[2022/08/30 10:36:10][INFO]   1 ns   : 1 Nanosecond (0.000000001 sec)
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // * Diagnostic Output - MemoryDiagnoser *
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] 
[2022/08/30 10:36:10][INFO] // ***** BenchmarkRunner: End *****
@LoopedBard3 LoopedBard3 added bug Something isn't working pipeline blocker issue blocks perf pipeline stays green. This be considered high priority. labels Aug 30, 2022
@radical
Copy link
Member

radical commented Sep 20, 2022

This was fixed in #2601 .

@radical radical closed this as completed Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working pipeline blocker issue blocks perf pipeline stays green. This be considered high priority.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@radical @LoopedBard3