Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Thread-unsafety in tests #370

Open
katrinafyi opened this issue Mar 13, 2025 · 1 comment
Open

Thread-unsafety in tests #370

katrinafyi opened this issue Mar 13, 2025 · 1 comment

Comments

@katrinafyi
Copy link
Member

katrinafyi commented Mar 13, 2025 •
edited
Loading

There are test suites which fail sporadically. I do not think the test cases are significant here. I do not know if these have the same cause.

Locally

DSAMemoryRegionSystemTestsBAP:

- incorrect/nestedifglobal/clang:BAP *** FAILED *** (101 milliseconds)
  java.util.NoSuchElementException: key not found: %000003ad$0: LocalVar(load20, bv32) := MemoryLoad(Memory(mem, 64, 8), bvadd64(Register(R8, bv64), 52bv64), LittleEndian, 32)
  at scala.collection.MapOps.default(Map.scala:289)
  at scala.collection.MapOps.default$(Map.scala:288)
  at scala.collection.AbstractMap.default(Map.scala:420)
  at scala.collection.mutable.HashMap.apply(HashMap.scala:440)
  at analysis.RegionInjectorDSA.injectRegions$$anonfun$4$$anonfun$1(RegionInjector.scala:231)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
  at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
  at util.intrusive_list.IntrusiveList.foreach(IntrusiveList.scala:30)
  at analysis.RegionInjectorDSA.injectRegions$$anonfun$4(RegionInjector.scala:214)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
  at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
  at scala.collection.AbstractIterator.foreach(Iterator.scala:1303)
  at analysis.RegionInjectorDSA.injectRegions(RegionInjector.scala:213)
  at util.RunUtils$.staticAnalysis(RunUtils.scala:1063)
  at util.RunUtils$.$anonfun$20(RunUtils.scala:854)
  at scala.Option.map(Option.scala:242)
  at util.RunUtils$.loadAndTranslate(RunUtils.scala:853)
  at test_util.BASILTest.runBASIL(BASILTest.scala:56)
  at test_util.BASILTest.runBASIL$(BASILTest.scala:31)
  at DSAMemoryRegionSystemTestsBAP.runBASIL(SystemTests.scala:464)
  at SystemTests.runTest(SystemTests.scala:189)
  at SystemTests.runTest$(SystemTests.scala:27)
  at DSAMemoryRegionSystemTestsBAP.runTest(SystemTests.scala:464)
  at SystemTests.testFun$proxy1$1(SystemTests.scala:83)
  at SystemTests.runTests$$anonfun$1$$anonfun$1$$anonfun$1(SystemTests.scala:84)
  at org.scalatest.Transformer.apply$$anonfun$1(Transformer.scala:22)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:31)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:21)
  at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:206)
  at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
  at org.scalatest.TestSuite.withFixture$(TestSuite.scala:138)
  at DSAMemoryRegionSystemTestsBAP.test_util$TestCustomisation$$super$withFixture(SystemTests.scala:464)
  at test_util.TestCustomisation.invokeTest$1(TestCustomisation.scala:56)
  at test_util.TestCustomisation.withFixture(TestCustomisation.scala:63)
  at test_util.TestCustomisation.withFixture$(TestCustomisation.scala:17)
  at DSAMemoryRegionSystemTestsBAP.withFixture(SystemTests.scala:464)
  at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:212)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTest$$anonfun$1(AnyFunSuiteLike.scala:216)
  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:216)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:47)
  at org.scalatest.funsuite.AnyFunSuite.runTest(AnyFunSuite.scala:1563)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTests$$anonfun$1(AnyFunSuiteLike.scala:249)
  at org.scalatest.SuperEngine.traverseSubNodes$1$$anonfun$1(Engine.scala:413)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.List.foreach(List.scala:334)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:429)
  at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:249)
  at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:47)
  at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)
  at org.scalatest.Suite.run(Suite.scala:1114)
  at org.scalatest.Suite.run$(Suite.scala:564)
  at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)
  at org.scalatest.funsuite.AnyFunSuiteLike.run$$anonfun$1(AnyFunSuiteLike.scala:253)
  at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
  at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:253)
  at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:47)
  at org.scalatest.funsuite.AnyFunSuite.run(AnyFunSuite.scala:1563)
  at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:47)
  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  at java.base/java.lang.Thread.run(Thread.java:840)

https://github.com/UQ-PAC/BASIL/actions/runs/13827512001/job/38685075601

DifferentialAnalysisTest:

- analysis_differential:basic_lock_security_read/clang_pic:GTIRB *** FAILED *** (72 milliseconds)
  java.util.NoSuchElementException: key not found: 1920_0_0: LocalVar(load23, bv64) := MemoryLoad(Memory(mem, 64, 8), bvadd64(Register(R8, bv64), 4040bv64), LittleEndian, 64)
  at scala.collection.MapOps.default(Map.scala:289)
  at scala.collection.MapOps.default$(Map.scala:288)
  at scala.collection.AbstractMap.default(Map.scala:420)
  at scala.collection.mutable.HashMap.apply(HashMap.scala:440)
  at analysis.data_structure_analysis.LocalPhase.visit(LocalPhase.scala:287)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation$$anonfun$1(LocalPhase.scala:135)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.Set$Set1.foreach(Set.scala:177)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation(LocalPhase.scala:135)
  ...

https://github.com/UQ-PAC/BASIL/actions/runs/13828662504/job/38688112951

SimplifyMemorySystemTests:

- correct/basic_lock_read/clang_pic:GTIRB *** FAILED *** (82 milliseconds)
  java.util.NoSuchElementException: key not found: 1920_0_1: Register(R8, bv64) := LocalVar(load0, bv64)
  at scala.collection.MapOps.default(Map.scala:289)
  at scala.collection.MapOps.default$(Map.scala:288)
  at scala.collection.AbstractMap.default(Map.scala:420)
  at scala.collection.mutable.HashMap.apply(HashMap.scala:440)
  at analysis.data_structure_analysis.LocalPhase.visit(LocalPhase.scala:258)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation$$anonfun$1(LocalPhase.scala:135)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.Set$Set1.foreach(Set.scala:177)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation(LocalPhase.scala:135)
  ...

To try and encounter the bugs yourself, the best way is to run the suites in a loop until they fail:

 while ./scripts/scalatest.sh -eGDCEFXK -P4 -l test_util.tags.DisabledTest >/dev/null ; do echo boop; done
@katrinafyi
Copy link
Member Author 

SimplifyMemorySystemTests:

- correct/basic_lock_security_read/clang_pic:BAP *** FAILED *** (51 milliseconds)
  java.util.NoSuchElementException: key not found: %0000036b$1: Register(R8, bv64) := LocalVar(load20, bv64)
  at scala.collection.MapOps.default(Map.scala:289)
  at scala.collection.MapOps.default$(Map.scala:288)
  at scala.collection.AbstractMap.default(Map.scala:420)
  at scala.collection.mutable.HashMap.apply(HashMap.scala:440)
  at analysis.data_structure_analysis.LocalPhase.visit(LocalPhase.scala:258)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation$$anonfun$1(LocalPhase.scala:135)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.Set$Set1.foreach(Set.scala:177)
  at analysis.data_structure_analysis.LocalPhase.visitPointerArithmeticOperation(LocalPhase.scala:135)
  at analysis.data_structure_analysis.LocalPhase.visit(LocalPhase.scala:317)
  at analysis.data_structure_analysis.LocalPhase.analyze$$anonfun$1(LocalPhase.scala:377)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.List.foreach(List.scala:334)
  at analysis.data_structure_analysis.LocalPhase.analyze(LocalPhase.scala:377)
  at analysis.data_structure_analysis.DataStructureAnalysis.analyze$$anonfun$1(DataStructureAnalysis.scala:86)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  at scala.collection.immutable.Set$Set1.foreach(Set.scala:177)
  at analysis.data_structure_analysis.DataStructureAnalysis.analyze(DataStructureAnalysis.scala:76)
  at util.RunUtils$.staticAnalysis(RunUtils.scala:1049)
  at util.RunUtils$.$anonfun$20(RunUtils.scala:854)
  at scala.Option.map(Option.scala:242)
  at util.RunUtils$.loadAndTranslate(RunUtils.scala:853)
  at test_util.BASILTest.runBASIL(BASILTest.scala:56)
  at test_util.BASILTest.runBASIL$(BASILTest.scala:31)
  at SimplifyMemorySystemTests.runBASIL(SystemTests.scala:380)
  at SystemTests.runTest(SystemTests.scala:189)
  at SystemTests.runTest$(SystemTests.scala:27)
  at SimplifyMemorySystemTests.runTest(SystemTests.scala:380)
  at SystemTests.testFun$proxy1$1(SystemTests.scala:83)
  at SystemTests.runTests$$anonfun$1$$anonfun$1$$anonfun$1(SystemTests.scala:84)
  at org.scalatest.Transformer.apply$$anonfun$1(Transformer.scala:22)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:31)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:21)
  at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:206)
  at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
  at org.scalatest.TestSuite.withFixture$(TestSuite.scala:138)
  at SimplifyMemorySystemTests.test_util$TestCustomisation$$super$withFixture(SystemTests.scala:380)
  at test_util.TestCustomisation.invokeTest$1(TestCustomisation.scala:56)
  at test_util.TestCustomisation.withFixture(TestCustomisation.scala:63)
  at test_util.TestCustomisation.withFixture$(TestCustomisation.scala:17)
  at SimplifyMemorySystemTests.withFixture(SystemTests.scala:380)

@katrinafyi katrinafyi mentioned this issue Mar 14, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@katrinafyi