dexpatcher / multidexlib2 Goto Github PK

Hey, is there a documentation that makes it easier for me to use dexlib2? Without documentation it makes me feel very difficult.

For example, I want to delete a class in dex, and get the smali file of a class, or modify the class by bytes.

Can you help me, thank you!

Zip open failed: Failure to verify dex file '/data/app/xxxxxxx-aV1pVhOcvp3FyS-LFU69rQ==/base.apk!classes2.dex': outs_size (7d) > registers_size (4d)

Hi Lanchon,

would you mind to update dexlib's version to 2.4.0?

Hi,

I just wanna shift the discussion to this place as the issues are related to multidexlib2. I tried to use the library as follows:

            File apkFile = new File(apkPath);

            // decode the APK file using apktool d -s
            decodedAPKPath = Utility.decodeAPK(apkFile);

            MultiDexContainer<? extends DexBackedDexFile> apk =
                    MultiDexIO.readMultiDexContainer(true, new File(decodedAPKPath),
                            new BasicDexFileNamer(), null, null);

            // instruments the dex files, e.g. adds additional methods, etc.
            apk.getDexEntryNames().forEach(dexFile -> {
                try {
                    instrument(apk.getEntry(dexFile).getDexFile(), dexFile, exclusionPattern);
                } catch (IOException e) {
                    LOGGER.warn("Failure loading dexFile");
                }
            });

            // add flags to manifest, insert entire smali class, etc

            // build the APK using apktool
            Utility.buildAPK(decodedAPKPath, outputAPKFile);

I originally tried to specify directly the APK file, but this failed with an exception in this ByteStreamHack class. Since I need to decode the APK anyways, this doesn't really matter. The APK contains two dex files (classes.dex and classes2.dex) where the former has 64693 method references and the latter solely ~ 3k. Then, my instrumentation process follows, which inserts among other things missing lifecycle methods in every activity/fragment. This, in turn leads to a classes.dex file that exceeds the method reference limit. The instrumentation procedure is similar to the following: https://gist.github.com/JesusFreke/6945806 btw. I basically request the MutableMethodImplementation objects and call on them addInstruction(), replaceInstruction(), etc. I also insert entire methods by creating such objects on my own. At the end of each call to instrument(), my code executes the following:

    // the file path refers to the decoded APK path (apktool -d, see above)
    // the classes list specifies all classes that should belong into the dex file (collected during instrumentation)
    public static void writeToDexFile(String filePath, List<ClassDef> classes, int opCode) throws IOException {

        DexFile dexFile = new DexFile() {
            @Nonnull
            @Override
            public Set<? extends ClassDef> getClasses() {
                return new AbstractSet<ClassDef>() {
                    @Nonnull
                    @Override
                    public Iterator<ClassDef> iterator() {
                        return classes.iterator();
                    }

                    @Override
                    public int size() {
                        return classes.size();
                    }
                };
            }

            @Nonnull
            @Override
            public Opcodes getOpcodes() {
                return Opcodes.forApi(opCode);
            }
        };
        MultiDexIO.writeDexFile(true, new File(filePath), new BasicDexFileNamer(),
                dexFile, DexIO.DEFAULT_MAX_DEX_POOL_SIZE, null);
    }

I have to specify here again a directory (a file is not working, which seems reasonable since the original classes.dex should be split into two dex files, as the method reference limit is hit). Although no exception occurrs, the resulting APK only contains a single classes.dex file, which refers to the 'classes2.dex' file of the original APK. I inspected the directory after the first instrumentation round and it contained the files classes.dex and classes2.dex as expected. However, somehow the instrumentation of the original classes2.dex overwrites the newly created dex files. In particular, the second call to MultiDexIO.writeDexFile() is responsible for this. Moreover, the MultiDexContainer object created by 'MultiDexIO.readMultiDexContainer()' is in an invalid state at this point. Iterating over the dex entries for example causes problems since the underlying dex files are not present anymore.
Any suggestions? Are two consecutive writes via MultiDexIO.writeDexFile() to the same directory not allowed? Do I need to specify another DexFileNamer object in this case?

Hi @Lanchon,

would you mind to point the dexlib2 dependency to the new smali repository at https://github.com/google/smali? Also apktool made this change in order to receive new updates.

Thanks in advance.

Hey there,

i'm currently trying to build a simple deobfuscator which is able to rename a method and all of it references.

I already investigated in dexlib2 and mdexlib2 and was successful at patching a method name itself. Now i'm a little stuck while trying to find a efficient way of finding all method references for the renamed method. I already found your way of patching method references but finding all calls to one single class method seems more difficult. Do you have a simple approach to it?

Thank you in advance!

java.lang.RuntimeException: Unable to instantiate application com.lynx.Application: java.lang.ClassNotFoundException: Didn't find class "com.lynx.Application:" on path: DexPathList[[zip

Looks like the library did not detect this as a start up class and put it in a secondary dex file.

The latest git master of smali/baksmali added support for new Android versions (up to dex version 39).
Could you please update multidexlib2 to support the same?

PS: Now there is mapDexVersionToApi.

Hi @Lanchon,

assume I need to add a JAR as a dependency to an existing APK. The first thing is to convert it to a .dex file using dx. That's fine, but I cannot simply place the .dex file alongside the original classes.dex file(s) and re-build the APK. I suppose the main dex file encodes some information to support multiple dex files. The current workaround is to convert both dex files to their smali representation and merge those folders manually and re-build the APK afterwards. That only works as long as there is enough space in the main .dex file. I thought about using multidexlib2, which can inherently handle multi-dex, but what happens when a potential conflict arises between those .dex files? It could happen that certain classes appear multiple times, because they are real duplicates or are different versions of the same class. Is multidexlib2 able to handle such conflicts on a best effort basis?

Thanks in advance!

Hi @Lanchon

Would you please consider releasing the library with more relaxed license, like LGPLv3 or even Apache-2?

Unfortunately, GPLv3 seriously limits usage of your work.

Thanks

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.io.ByteStreams.toByteArray(Ljava/io/InputStream;J)[B
	at com.google.common.io.ByteStreamsHack.toByteArray(ByteStreamsHack.java:30)
	at lanchon.multidexlib2.RawDexIO.readRawDexFile(RawDexIO.java:54)
	at lanchon.multidexlib2.ZipFileDexContainer.<init>(ZipFileDexContainer.java:48)
	at lanchon.multidexlib2.MultiDexIO.readMultiDexContainer(MultiDexIO.java:61)
	at lanchon.multidexlib2.MultiDexIO.readMultiDexContainer(MultiDexIO.java:48)
	at lanchon.multidexlib2.MultiDexIO.readMultiDexContainer(MultiDexIO.java:39)

reference: #7 (comment)

Reading the following APK https://github.com/the-themis-benchmarks/home/blob/master/open-event-attendee-android/open-event-attendee-android-0.5-%232198.apk with multidexlib2 leads to the following exception:

Exception in thread "main" lanchon.multidexlib2.DuplicateTypeException: L$r8$java8methods$utility$Long$hashCode$IJ;
        at lanchon.multidexlib2.MultiDexContainerBackedDexFile.<init>(MultiDexContainerBackedDexFile.java:42)
        at lanchon.multidexlib2.MultiDexIO.readDexFile(MultiDexIO.java:34)
        at de.uni_passau.fim.auermich.android_analysis.Main.main(Main.java:91)

The APK contains twice the class L$r8$java8methods$utility$Long$hashCode$IJ; in two different smali folders. I could resolve the issue by simply removing one of them, but shouldn't multidexlib2 maybe simply ignore any duplicate and only drop a warning?