Comments (10)
Yep! It should run on android too, but as you've noticed, the LazySodiumJava library isn't designed for Android - this is because it is simply a wrapper around libsodium that is actually written in C.
Fortunately there's an easy drop-in replacement as a fix - https://github.com/terl/lazysodium-android ;) So you'll need to change the dependency and possibly the class name, to LazySodiumAndroid but the functions should remain the same!
from unapkm.
You probably mean this line:
LazySodiumJava lazySodium = new LazySodiumJava(new SodiumJava());
I changed it to this:
LazySodiumAndroid lazySodium = new LazySodiumAndroid(new SodiumAndroid());
Also had to changed the dependencies a bit.
Attached here the output project:
Seems to work fine, but for some reason the output zip couldn't be opened via a file manager I use (Total Commander), yet it worked fine when I opened it via the PC.
Any idea why?
Also, it seems that before the loop of reading, it takes quite some time to initialize. Is the decrypting/decoding so intense?
On a relatively low end device (Xiaomi Redmi 8), it took 4 seconds in total to handle a 23MB file as input (has 18 files inside), and almost all of it was the part before the loop. I guess it can be worse for larger APKM files. Is there a way to optimize it? It's a huge time compared to a simple ZIP file.
Maybe you should offer a listener for the status of the reading, to allow to show progress and report errors.
from unapkm.
Yep, that line sounds about right :)
The output zip isn't a proper zip file as it's missing the end-of-central-directory signature, but this is simply what was encoded into the apkm file pre-crypto. It is still supported by ZipInputStream
however so you should still be able to parse it with java or many of the other options
The 4 second delay is likely the hash generation function (the "memory-hard, CPU-intensive hash function" described at https://libsodium.gitbook.io/doc/password_hashing/default_phf ) - but thankfully all this only needs to be done once per apkm file (and alas is necessary) :) It shouldn't be any worse on larger APKM files because it's only the small number of bytes at the very start of the file that are put through this hash function to derive the secret key, not the whole file.
from unapkm.
- Is there a way to fix the output file so that file manager apps would be able to open it?
Using ZipFile or ZipInputStream , it works fine for me, so I don't get why Total Commander app couldn't handle it:
try {
Log.d("AppLog", "opening using ZipInputStream:")
ZipInputStream(FileInputStream(outputFile)).use {
while(true){
val entry = it.nextEntry?:return@use
Log.d("AppLog", "${entry.name} - ${entry.size}")
}
}
Log.d("AppLog", "opening using ZipFile:")
val zipFile = ZipFile(outputFile)
for (entry in zipFile.entries()) {
Log.d("AppLog", "${entry.name} - ${entry.size}")
}
}
catch (e:Exception){
}
-
As for the 4 seconds, is there a way to minimize it?
-
Is there a way to have this 4 seconds init only once per APKM file? For example, suppose I check some content of the stream, do something else, and then I want to handle the stream again from beginning. Maybe there is a way to cache what's created in this 4 seconds, to avoid having another 4 seconds? If it's possible, I think you should change the static functions so that it would be a part of a normal class, that has the caching per file. The CTOR will be minimal, of course, and won't require any handling of files there.
from unapkm.
- Yep you can feed the ZipInputStream into a ZipOutputStream and use that to save to a file which should fix this error
2 and 3. You should be able to minimise it using the convenience header functions I've added in - you can call processHeader(i, lazySodium, true)
to get your header the first time round, and then next time you open the stream you can just use processHeader(i, lazySodium, false) to skip past the header without doing the crypto stuff and use the previously cached header to process it in decryptStream(i, h, lazySodium);
- this header is file specific so beware
from unapkm.
- OK I tried this:
UnApkm.decryptStream(FileInputStream(inputFile)).copyTo(ZipOutputStream(FileOutputStream(outputFile)))
This causes this exception:
2020-03-27 14:21:05.356 7434-7497/com.lb.apkmtest E/AndroidRuntime: FATAL EXCEPTION: Thread-2
Process: com.lb.apkmtest, PID: 7434
java.util.zip.ZipException: no current ZIP entry
at java.util.zip.ZipOutputStream.write(ZipOutputStream.java:328)
at kotlin.io.ByteStreamsKt.copyTo(IOStreams.kt:108)
at kotlin.io.ByteStreamsKt.copyTo$default(IOStreams.kt:103)
at com.lb.apkmtest.MainActivity$onGotPermissions$1.invoke(MainActivity.kt:51)
at com.lb.apkmtest.MainActivity$onGotPermissions$1.invoke(MainActivity.kt:19)
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)
I think the function already created a zip content, so I think it doesn't make sense to provide it here.
So I used this instead:
UnApkm.decryptStream(FileInputStream(inputFile)).copyTo(FileOutputStream(outputFile))
But then, again, the file can't be opened there.
Attached project here:
ApkmTest.zip
2.3. I don't think I did it well, or you. Something is weird here.
This is what I did:
Log.d("AppLog", "getting header")
val lazySodiumAndroid = LazySodiumAndroid(SodiumAndroid())
val header = UnApkm.processHeader(FileInputStream(inputFile), lazySodiumAndroid, true)
Log.d("AppLog", "done getting header. handling content")
UnApkm.processHeader(FileInputStream(inputFile), lazySodiumAndroid, false)
UnApkm.decryptStream(FileInputStream(inputFile), header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
Log.d("AppLog", "done handling content. Handle it again:")
outputFile.delete()
UnApkm.processHeader(FileInputStream(inputFile), lazySodiumAndroid, false)
UnApkm.decryptStream(FileInputStream(inputFile), header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
Log.d("AppLog", "done handling content again")
The exception I get:
2020-03-27 14:37:24.907 9151-9209/com.lb.apkmtest W/System.err: java.io.IOException: decrypto error
2020-03-27 14:37:24.907 9151-9209/com.lb.apkmtest W/System.err: at com.lb.apkmtest.UnApkm.lambda$decryptStream$0(UnApkm.java:145)
2020-03-27 14:37:24.908 9151-9209/com.lb.apkmtest W/System.err: at com.lb.apkmtest.-$$Lambda$UnApkm$SQ8gROLqltIDjnsz2Wg8Bb946lk.run(Unknown Source:8)
2020-03-27 14:37:24.908 9151-9209/com.lb.apkmtest W/System.err: at java.lang.Thread.run(Thread.java:764)
2020-03-27 14:37:24.908 9151-9204/com.lb.apkmtest D/AppLog: done handling content. Handle it again:
2020-03-27 14:37:24.919 9151-9210/com.lb.apkmtest W/System.err: java.io.IOException: decrypto error
.
2020-03-27 14:37:24.919 9151-9210/com.lb.apkmtest W/System.err: at com.lb.apkmtest.UnApkm.lambda$decryptStream$0(UnApkm.java:145)
2020-03-27 14:37:24.919 9151-9210/com.lb.apkmtest W/System.err: at com.lb.apkmtest.-$$Lambda$UnApkm$SQ8gROLqltIDjnsz2Wg8Bb946lk.run(Unknown Source:8)
2020-03-27 14:37:24.919 9151-9210/com.lb.apkmtest W/System.err: at java.lang.Thread.run(Thread.java:764)
Attached project :
ApkmTest.zip
I really suggest to decouple the implementation of LazySodium
, so that it could be used on Android too, easily.
from unapkm.
You are creating too many FileInputStreams - the crypted data is stored after the header, and by creating a new stream you are trying to decrypt the header instead of the ciphertext. Processing the header without computation on the same stream skips ahead to the ciphertext.
Log.d("AppLog", "getting header")
val lazySodiumAndroid = LazySodiumAndroid(SodiumAndroid())
FileInputStream fis = FileInputStream(inputFile);
val header = UnApkm.processHeader(fis, lazySodiumAndroid, true)
Log.d("AppLog", "done getting header. handling content")
UnApkm.decryptStream(fis, header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
Log.d("AppLog", "done handling content. Handle it again:")
outputFile.delete()
fis.close();
fis = FileInputStream(inputFile);
UnApkm.processHeader(fis, lazySodiumAndroid, false)
UnApkm.decryptStream(fis, header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
fis.close();
Log.d("AppLog", "done handling content again")
from unapkm.
Oh right I forgot to close.
I wanted just the header in the beginning, to show that this one takes most of the time, and that later it's almost nothing, so I need to use a new InputStream 3 times : header, content, and again content.
Here's the new code for this:
Log.d("AppLog", "getting header")
val lazySodiumAndroid = LazySodiumAndroid(SodiumAndroid())
val header = FileInputStream(inputFile).use { UnApkm.processHeader(it, lazySodiumAndroid, true) }
Log.d("AppLog", "done getting header. handling content")
FileInputStream(inputFile).use {
UnApkm.processHeader(it, lazySodiumAndroid, false)
UnApkm.decryptStream(it, header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
}
Log.d("AppLog", "done handling content. Handle it again:")
outputFile.delete()
FileInputStream(inputFile).use {
UnApkm.processHeader(it, lazySodiumAndroid, false)
UnApkm.decryptStream(it, header, lazySodiumAndroid).copyTo(FileOutputStream(outputFile))
}
Log.d("AppLog", "done handling content again")
This works fine, but it shows in the logs that each of those took a long time:
Also, BTW, the file manager still can't handle the output zip file.
from unapkm.
I've fixed the ZIP output bug in the latest commit and release.
How weird! Please do look into if you can see any other bottlenecks - you can always cache the outputstream into a byte array to keep reusing it so you only need to decrypt it once!
But yes crypto operations are cpu/mem heavy.
from unapkm.
Zip file still can't be opened (via Total Commander app), and the time to handle the content after getting the header is still about as long as getting the header. However, for some reason, after the first handling of the content, the second one takes a short time.
Saving files content into the RAM is very efficient but very problematic in case the content is too large. I also don't remember if ByteArray on Android has come into the native memory instead of heap. I know Bitmap data has, but I don't remember if ByteArray has. If it's part of the heap, it's even more problematic, because heap size might be too small.
In any case, I suggest not to do it in the library, or if you insist: to provide it as a parameter of how much memory is allowed for it.
from unapkm.
Related Issues (12)
- Request: split UnApkm function to use streams instead of files HOT 6
- Question: what is "memLimit" ? HOT 12
- Request: apply some adjustments and suggestions from the IDE HOT 10
- Request: Make another module for Android HOT 3
- Cannot find tempfiles to properly work. HOT 2
- Fdroid HOT 2
- Please target JAVA version 52.0 instead of 53.0 HOT 4
- "incorrect algo" error HOT 1
- Request: put license to the repository HOT 5
- Question: how come I can't re-use the header? HOT 1
- Decrypto error HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unapkm.