Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Soot-cmd version crash in merge multidex mode #786

Open
Dakyz opened this issue Dec 11, 2024 · 3 comments
Open

Soot-cmd version crash in merge multidex mode #786

Dakyz opened this issue Dec 11, 2024 · 3 comments

Comments

@Dakyz
Copy link

Dakyz commented Dec 11, 2024

I run FlowDroid on this APK file -
123.zip (APK inside zip archive). I used soot-cmd (tried 2.14.1, and 2.13 but this bug appears on both of them).

Run FlowDroid with such args.

java -jar soot.jar -a 123.apk -p platform-tools -s SourcesAndSinks.txt -d -o result.xml

When I try without -d it runs successfully, but it checks only 1 dex - that's not good. Tried another APKs with the same platform-tools and SourcesAndSinks.txt so everything was fine except this APK.

Add logs of this crash:

[main] INFO soot.jimple.infoflow.android.SetupApplication - Collecting callbacks and building a callgraph took 338 seconds
[main] INFO soot.jimple.infoflow.android.SetupApplication - Running data flow analysis on PhoneMaster_new.apk with 60 sources and 73 sinks...
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Implicit flow tracking is NOT enabled
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Exceptional flow tracking is enabled
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Running with a maximum access path length of 5
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Using path-agnostic result collection
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Recursive access path shortening is enabled
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Taint analysis enabled: true
[main] INFO soot.jimple.infoflow.InfoflowConfiguration - Using alias algorithm FlowSensitive
[main] INFO soot.jimple.infoflow.memory.MemoryWarningSystem - Registered a memory warning system for 3,645 MiB
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Callgraph construction took 0 seconds
[main] INFO soot.jimple.infoflow.codeOptimization.InterproceduralConstantValuePropagator - Removing side-effect free methods is disabled
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Dead code elimination took 3.6065567 seconds
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Callgraph has 113305 edges
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Starting Taint Analysis
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Using context- and flow-sensitive solver
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Using context- and flow-sensitive solver
[main] WARN soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Running with limited join point abstractions can break context-sensitive path builders
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Looking for sources and sinks...
[main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Source lookup done, found 59 sources and 593 sinks.
[FlowDroid] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null
java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:904)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4016)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4996)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:130)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:153)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:51)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:208)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:51)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.runInternal(IFDSSolver.java:750)
        at soot.jimple.infoflow.solver.fastSolver.LocalWorklistTask.run(LocalWorklistTask.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:842)
[FlowDroid] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null
java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:904)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4016)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4996)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:130)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:153)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:51)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:208)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:51)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.runInternal(IFDSSolver.java:750)
        at soot.jimple.infoflow.solver.fastSolver.LocalWorklistTask.run(LocalWorklistTask.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:842)
Exception in thread "FlowDroid" [main] INFO soot.jimple.infoflow.memory.MemoryWarningSystem - Shutting down the memory warning system...
Exception in thread "FlowDroid" java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:904)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4016)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4996)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:130)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:153)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:51)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:208)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:51)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.runInternal(IFDSSolver.java:750)
        at soot.jimple.infoflow.solver.fastSolver.LocalWorklistTask.run(LocalWorklistTask.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:842)
java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:904)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4016)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4996)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:130)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:153)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:51)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:208)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:51)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.runInternal(IFDSSolver.java:750)
        at soot.jimple.infoflow.solver.fastSolver.LocalWorklistTask.run(LocalWorklistTask.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:842)
[main] ERROR soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Exception during data flow analysis
java.lang.RuntimeException: There were exceptions during IFDS analysis. Exiting.
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.runExecutorAndAwaitCompletion(IFDSSolver.java:263)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.awaitCompletionComputeValuesAndShutdown(IFDSSolver.java:230)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.solve(IFDSSolver.java:202)
        at soot.jimple.infoflow.AbstractInfoflow.runTaintAnalysis(AbstractInfoflow.java:1177)
        at soot.jimple.infoflow.AbstractInfoflow.runAnalysis(AbstractInfoflow.java:875)
        at soot.jimple.infoflow.AbstractInfoflow.runAnalysis(AbstractInfoflow.java:808)
        at soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow.runAnalysis(SetupApplication.java:1358)
        at soot.jimple.infoflow.android.SetupApplication.processEntryPoint(SetupApplication.java:1656)
        at soot.jimple.infoflow.android.SetupApplication.runInfoflow(SetupApplication.java:1585)
        at soot.jimple.infoflow.android.SetupApplication.runInfoflow(SetupApplication.java:1532)
        at soot.jimple.infoflow.cmd.MainClass.run(MainClass.java:362)
        at soot.jimple.infoflow.cmd.MainClass.main(MainClass.java:260)
Caused by: java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:904)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4016)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4040)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4989)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4996)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:130)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:153)
        at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.isExitStmt(AbstractJimpleBasedICFG.java:51)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:208)
        at soot.jimple.infoflow.solver.cfg.InfoflowCFG.isExitStmt(InfoflowCFG.java:51)
        at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.runInternal(IFDSSolver.java:750)
        at soot.jimple.infoflow.solver.fastSolver.LocalWorklistTask.run(LocalWorklistTask.java:27)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:842)
[main] INFO soot.jimple.infoflow.android.SetupApplication - Found 0 leaks from 0 sources
@Dakyz Dakyz changed the title Crash in cmd version in merge multidex mode Soot-cmd version crash in merge multidex mode Dec 11, 2024
@StevenArzt
Copy link
Member

The problem is not with Multidex support. The code that triggers the bug just seems to be in one of the additional dex files by chance.

Apparently, the a method is added (or for some reason replaced) after the ICFG has been constructed. We have had such bugs in the past and they are nasty to find. For the time being, I suggest that you patch AbstractJimpleBasedICFG.isExitStmt to check whether body is null and return false if this is the case. Only continue to call getOrCreateUnitGraph if you have a valid, i.e., non-null, body.

That fix hides the underlying problem and may lead to false negatives at least in theory. In practice, it's hard to tell whether these cases are relevant. Still, the proposed quickfix allows you to move on - it's a five minute change as opposed to a multi-hour debugging session.

You might need to check the other places where getOrCreateUnitGraph is called inside getOrCreateUnitGraph as well.

@Dakyz
Copy link
Author

Dakyz commented Dec 11, 2024

well, ofc I try to patch this and check

I suggest that you patch AbstractJimpleBasedICFG.isExitStmt to check whether body is null and return false if this is the case

but I hope you understand that such kind of stub is not good...

it's a five minute change as opposed to a multi-hour debugging session.

thank you for answer and suggestion at least.

@StevenArzt
Copy link
Member

I know that my proposed quickfix is not a replacement for a proper solution. However, it allows you to get rid of the problem for now since I don't have the time to debug the root cause right now. We have tons of things going on in the area of data flow research that may have great impact on FlowDroid in my group, and this bug is not top priority unless we see it in more than a handful of apps.

@wyatt-feng wyatt-feng mentioned this issue Dec 15, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StevenArzt @Dakyz