ucr-riple / androidslicer Goto Github PK

Hi, I tried to compile and run "instrumenter" from the source code (instead of running instrumenter.sh).

By setting Instrumenter's main as a default function, I used this arguments.
com.twitter.android -w -allow-phantom-refs -process-multiple-dex -force-android-jar <sdk_path>/android-28/android.jar -src-prec apk -output-format dex -process-dir <my_path>/AndroidSlicer/tool/Twitter_v7.93.2-release.50_apkpure.com.apk

The app I used is from Google Play Store for version 7.93.2
After that, there is no error for a while, and it ran for like 5 minutes.

At the end, the program dies because of GC
I found that the paper used Twitter as a benchmark. I am curious how to instrument it.
I tried it with a small example, it could be done.
It would be really appreciated if you help me to figure this out.

Below is the error message.

Soot started on Sun Jul 28 16:59:22 PDT 2019
[Thread-9] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-9" java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.PackManager.writeOutput(PackManager.java:716)
at soot.PackManager.writeDexOutput(PackManager.java:584)
at soot.PackManager.writeOutput(PackManager.java:567)
at soot.Main.run(Main.java:271)
at soot.Main.main(Main.java:141)
at org.ucr.ds.cd.utilities.Instrumenter.main(Instrumenter.java:407)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
java.lang.OutOfMemoryError: GC overhead limit exceeded
at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86)
at soot.jimple.StringConstant.apply(StringConstant.java:63)
at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489)
at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242)
at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511)
at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174)
at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083)
at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656)
at soot.toDex.DexPrinter.add(DexPrinter.java:1646)
at soot.PackManager.writeClass(PackManager.java:1096)
at soot.PackManager.lambda$writeOutput$1(PackManager.java:699)
at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Ouuups... something went wrong! Sorry about that.
Follow these steps to fix the problem:
1.) Are you sure you used the right command line?
Click here to double-check:
https://github.com/Sable/soot/wiki/Options-and-JavaDoc

2.) Not sure whether it's a bug? Feel free to discuss
the issue on the Soot mailing list:
https://github.com/Sable/soot/wiki/Getting-help

3.) Sure it's a bug? Click this link to report it.
https://github.com/Sable/soot/issues/new?title=java.lang.RuntimeException+when+...&body=Steps+to+reproduce%3A%0A1.%29+...%0A%0AFiles+used+to+reproduce%3A+%0A...%0A%0ASoot+version%3A+%3Cpre%3Etrunk%3C%2Fpre%3E%0A%0ACommand+line%3A%0A%3Cpre%3E-w+-allow-phantom-refs+-process-multiple-dex+-force-android-jar+%2Fhome%2Fchungha%2FAndroid%2FSdk%2Fplatforms%2Fandroid-28%2Fandroid.jar+-src-prec+apk+-output-format+dex+-process-dir+%2Fhome%2Fchungha%2Fwork%2FAndroidSlicer%2Ftool%2FTwitter_v7.93.2-release.50_apkpure.com.apk%3C%2Fpre%3E%0A%0AMax+Memory%3A%0A%3Cpre%3E3775MB%3C%2Fpre%3E%0A%0AStack+trace%3A%0A%3Cpre%3Ejava.lang.RuntimeException%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A716%29%0A%09at+soot.PackManager.writeDexOutput%28PackManager.java%3A584%29%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A567%29%0A%09at+soot.Main.run%28Main.java%3A271%29%0A%09at+soot.Main.main%28Main.java%3A141%29%0A%09at+org.ucr.ds.cd.utilities.Instrumenter.main%28Instrumenter.java%3A407%29%0ACaused+by%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.toDex.ConstantVisitor.caseStringConstant%28ConstantVisitor.java%3A86%29%0A%09at+soot.jimple.StringConstant.apply%28StringConstant.java%3A63%29%0A%09at+soot.toDex.StmtVisitor.caseAssignStmt%28StmtVisitor.java%3A489%29%0A%09at+soot.jimple.internal.JAssignStmt.apply%28JAssignStmt.java%3A242%29%0A%09at+soot.toDex.DexPrinter.toInstructions%28DexPrinter.java%3A1511%29%0A%09at+soot.toDex.DexPrinter.toMethodImplementation%28DexPrinter.java%3A1174%29%0A%09at+soot.toDex.DexPrinter.toMethods%28DexPrinter.java%3A1083%29%0A%09at+soot.toDex.DexPrinter.addAsClassDefItem%28DexPrinter.java%3A656%29%0A%09at+soot.toDex.DexPrinter.add%28DexPrinter.java%3A1646%29%0A%09at+soot.PackManager.writeClass%28PackManager.java%3A1096%29%0A%09at+soot.PackManager.lambda%24writeOutput%241%28PackManager.java%3A699%29%0A%09at+soot.PackManager%24%24Lambda%242%2F1879083009.run%28Unknown+Source%29%0A%09at+java.util.concurrent.ThreadPoolExecutor.runWorker%28ThreadPoolExecutor.java%3A1149%29%0A%09at+java.util.concurrent.ThreadPoolExecutor%24Worker.run%28ThreadPoolExecutor.java%3A624%29%0A%09at+java.lang.Thread.run%28Thread.java%3A748%29%0A%3C%2Fpre%3E
Please be as precise as possible when giving us
information on how to reproduce the problem. Thanks!

Process finished with exit code 1

Hi,

I am sorry for many questions. I really wanted to make this work :p

So, the question is when which source file corresponds to preSlicer.jar and AndroidSlicer.jar?

In the source code I can see three main files from Instrumeneter class, TestInside class, and Slicer class.

I can see Instrumenter.java and Instrumenter.class are from Instrument class in source code.

But, I am not sure for AndroidSlicer.jar and preSlicer.jar

Could you let me know which source code is related with these jar files?

Also, could you let me know how to actually recompile/build jars to get the same jar files?

I really appreciate your answers :)

Hi,

I tried to run ./instrument.sh , it gives me this error.

Could you let me know how I should fix it?
(I updated jre bath)

Thank you !

if you closed adb logcat press enter

'com.example.motex'
Note: Instrumenter.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
pkg: com.example.motex
Soot started on Sun Jul 28 16:52:36 PDT 2019
Using '/home/chungha/Android/Sdk/platforms//android-28/android.jar' as android.jar
java.lang.NullPointerException
at soot.JastAddJ.Program.initPaths(Program.java:350)
at soot.SootResolver.(SootResolver.java:88)
at soot.Singletons.soot_SootResolver(Singletons.java:1456)
at soot.SootResolver.v(SootResolver.java:93)
at soot.Scene.tryLoadClass(Scene.java:713)
at soot.Scene.loadBasicClasses(Scene.java:1374)
at soot.Scene.loadNecessaryClasses(Scene.java:1453)
at soot.Main.run(Main.java:243)
at soot.Main.main(Main.java:147)
at Instrumenter.main(Instrumenter.java:405)

Ouuups... something went wrong! Sorry about that.
Follow these steps to fix the problem:
1.) Are you sure you used the right command line?
Click here to double-check:
https://ssebuild.cased.de/nightly/soot/doc/soot_options.htm

2.) Not sure whether it's a bug? Feel free to discuss
the issue on the Soot mailing list:
https://github.com/Sable/soot/wiki/Getting-help

3.) Sure it's a bug? Click this link to report it.
https://github.com/Sable/soot/issues/new?title=java.lang.NullPointerException+when+...&body=Steps+to+reproduce%3A%0A1.%29+...%0A%0AFiles+used+to+reproduce%3A+%0A...%0A%0ASoot+version%3A+%3Cpre%3Etrunk%3C%2Fpre%3E%0A%0ACommand+line%3A%0A%3Cpre%3E-w+-allow-phantom-refs+-process-multiple-dex+-android-jars+%2Fhome%2Fchungha%2FAndroid%2FSdk%2Fplatforms%2F+-src-prec+apk+-output-format+dex+-process-dir+ex1-1.apk%3C%2Fpre%3E%0A%0AMax+Memory%3A%0A%3Cpre%3E5120MB%3C%2Fpre%3E%0A%0AStack+trace%3A%0A%3Cpre%3Ejava.lang.NullPointerException%0A%09at+soot.JastAddJ.Program.initPaths%28Program.java%3A350%29%0A%09at+soot.SootResolver.%26%2360%3Binit%26%2362%3B%28SootResolver.java%3A88%29%0A%09at+soot.Singletons.soot_SootResolver%28Singletons.java%3A1456%29%0A%09at+soot.SootResolver.v%28SootResolver.java%3A93%29%0A%09at+soot.Scene.tryLoadClass%28Scene.java%3A713%29%0A%09at+soot.Scene.loadBasicClasses%28Scene.java%3A1374%29%0A%09at+soot.Scene.loadNecessaryClasses%28Scene.java%3A1453%29%0A%09at+soot.Main.run%28Main.java%3A243%29%0A%09at+soot.Main.main%28Main.java%3A147%29%0A%09at+Instrumenter.main%28Instrumenter.java%3A405%29%0A%3C%2Fpre%3E
Please be as precise as possible when giving us
information on how to reproduce the problem. Thanks!
chmod: cannot access 'sootOutput/ex1-1.apk': No such file or directory
signing sootOutput/ex1-1.apk to sootOutput/ex1-1.apk_signed.apk
java.nio.file.NoSuchFileException: sootOutput/ex1-1.apk
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:145)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1225)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:727)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:844)
at java.base/java.util.zip.ZipFile.(ZipFile.java:247)
at java.base/java.util.zip.ZipFile.(ZipFile.java:177)
at java.base/java.util.jar.JarFile.(JarFile.java:346)
at java.base/java.util.jar.JarFile.(JarFile.java:317)
at java.base/java.util.jar.JarFile.(JarFile.java:297)
at com.android.signapk.SignApk.main(SignApk.java:320)
chmod: cannot access 'ex1-1.apk_signed.apk': No such file or directory

Hi there,

nice work coming up with a slicer for Android apps.
However, I got some issues using it.

All files mentioned in the following are included here: test.zip

After Step 4 I found the following line in testApp.apk_signed.apk.logcat.txt:
11-09 11:36:51.183 19538 19538 I System.out: CALLBACK_SLC: SLICING: ZZZ-1ZZZde.foellix.aql.slicer.slicertestapp.TargetLeakZZZonCreateZZZ__inst__ZZZvirtualinvoke $r4.<android.telephony.SmsManager: void sendTextMessage(java.lang.String,java.lang.String,java.lang.String,android.app.PendingIntent,android.app.PendingIntent)>("+49 1234", null, $r3, null, null)
This line includes the desired statement that should be used as slicing criterion, however, after executing Step 5 I cannot find that statement anymore in testApp.apk_signed.apk.logcat.processed.txt.
Thus, I cannot proceed with the last step, since I cannot identify the intended slicing criterion. What am I missing?

Made three adaptions to the usage-instructions given:

• To get through step 3 I had to adapt the path to the rt.jar in instrumenter.sh,
• for signing I had to remove MANIFEST.MF from the APK generated by Soot. Added zip -d sootOutput/$1 "META-INF/MANIFEST.MF" to instrumenter.sh in line 18 to do so. Thereafter, I could successfully instrument, install and run the APK.
• adb logcat | grep SLICING > testApp.apk_signed.apk.logcat.txt gave me incomplete output (see last line in testApp_signed.apk.logcat_incomplete.txt). Thus, I used adb logcat -e "SLICING" > testApp.apk_signed.apk.logcat.txt which should do the same.

Looking forward to hearing from you!

Cheers,
FoelliX

Hi,

when I tried to sign the apk after instrumentation, it shows this error when I ran this command:
java -jar signapk.jar testkey.x509.pem testkey.pk8 sootOutput/ex1-1.apk ex1-1_signed.apk

Exception in thread "main" java.lang.NoClassDefFoundError: sun/misc/BASE64Encoder
at com.android.signapk.SignApk.addDigestsToManifest(SignApk.java:169)
at com.android.signapk.SignApk.main(SignApk.java:325)
Caused by: java.lang.ClassNotFoundException: sun.misc.BASE64Encoder
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 2 more

I couldn't find any source code for this jar so it is hard to figure out the problem.
Can I just sign the app in my own way? such as using jarsigner provided by bash command?
I am not sure if this affects the result.