marcobellaccini / pyaescrypt Goto Github PK
View Code? Open in Web Editor NEWA Python 3 module and script that uses AES256-CBC to encrypt/decrypt files and streams in AES Crypt file format (version 2).
License: Apache License 2.0
A Python 3 module and script that uses AES256-CBC to encrypt/decrypt files and streams in AES Crypt file format (version 2).
License: Apache License 2.0
I have 18 3.5KB files that I want to decrypt. This takes 18 seconds with pyAesCrypt.
I already did some profiling and found that 99.9% of the decryption time is spent in key = stretch(passw, iv1).
In stretch() there is a line that takes 53% of the time (it is called in a loop 8192 times):
passHash = hashes.Hash(hashes.SHA256(), backend=default_backend())
Can we speed this up by using other parameters or another implementation?
Here is the profile:
Total time: 18.0321 s
File: python/lib/python3.9/site-packages/pyAesCrypt/crypto.py
Function: decryptStream at line 293
Line # Hits Time Per Hit % Time Line Contents
==============================================================
293 @profile
294 def decryptStream(fIn, fOut, passw, bufferSize, inputLength):
295 # validate bufferSize
296 67 109.0 1.6 0.0 if bufferSize % AESBlockSize != 0:
297 raise ValueError("Buffer size must be a multiple of AES block size")
298
299 67 83.0 1.2 0.0 if len(passw) > maxPassLen:
300 raise ValueError("Password is too long.")
301
302 67 75.0 1.1 0.0 fdata = fIn.read(3)
303 # check if file is in AES Crypt format (also min length check)
304 67 110.0 1.6 0.0 if (fdata != bytes("AES", "utf8") or inputLength < 136):
305 raise ValueError("File is corrupted or not an AES Crypt "
306 "(or pyAesCrypt) file.")
307
308 # check if file is in AES Crypt format, version 2
309 # (the only one compatible with pyAesCrypt)
310 67 58.0 0.9 0.0 fdata = fIn.read(1)
311 67 58.0 0.9 0.0 if len(fdata) != 1:
312 raise ValueError("File is corrupted.")
313
314 67 45.0 0.7 0.0 if fdata != b"\x02":
315 raise ValueError("pyAesCrypt is only compatible with version "
316 "2 of the AES Crypt file format.")
317
318 # skip reserved byte
319 67 77.0 1.1 0.0 fIn.read(1)
320
321 # skip all the extensions
322 while True:
323 201 173.0 0.9 0.0 fdata = fIn.read(2)
324 201 157.0 0.8 0.0 if len(fdata) != 2:
325 raise ValueError("File is corrupted.")
326 201 152.0 0.8 0.0 if fdata == b"\x00\x00":
327 67 50.0 0.7 0.0 break
328 134 191.0 1.4 0.0 fIn.read(int.from_bytes(fdata, byteorder="big"))
329
330 # read external iv
331 67 56.0 0.8 0.0 iv1 = fIn.read(16)
332 67 50.0 0.7 0.0 if len(iv1) != 16:
333 raise ValueError("File is corrupted.")
334
335 # stretch password and iv
336 67 18006446.0 268752.9 99.9 key = stretch(passw, iv1)
337
338 # read encrypted main iv and key
339 67 124.0 1.9 0.0 c_iv_key = fIn.read(48)
340 67 73.0 1.1 0.0 if len(c_iv_key) != 48:
341 raise ValueError("File is corrupted.")
342
343 # read HMAC-SHA256 of the encrypted iv and key
344 67 56.0 0.8 0.0 hmac1 = fIn.read(32)
345 67 54.0 0.8 0.0 if len(hmac1) != 32:
346 raise ValueError("File is corrupted.")
347
348 # compute actual HMAC-SHA256 of the encrypted iv and key
349 134 1650.0 12.3 0.0 hmac1Act = hmac.HMAC(key, hashes.SHA256(),
350 67 373.0 5.6 0.0 backend=default_backend())
351 67 402.0 6.0 0.0 hmac1Act.update(c_iv_key)
352
353 # HMAC check
354 67 650.0 9.7 0.0 if hmac1 != hmac1Act.finalize():
355 raise ValueError("Wrong password (or file is corrupted).")
356
357 # instantiate AES cipher
358 134 1421.0 10.6 0.0 cipher1 = Cipher(algorithms.AES(key), modes.CBC(iv1),
359 67 371.0 5.5 0.0 backend=default_backend())
360 67 4768.0 71.2 0.0 decryptor1 = cipher1.decryptor()
361
362 # decrypt main iv and key
363 67 1981.0 29.6 0.0 iv_key = decryptor1.update(c_iv_key) + decryptor1.finalize()
364
365 # get internal iv and key
366 67 69.0 1.0 0.0 iv0 = iv_key[:16]
367 67 57.0 0.9 0.0 intKey = iv_key[16:]
368
369 # instantiate another AES cipher
370 134 925.0 6.9 0.0 cipher0 = Cipher(algorithms.AES(intKey), modes.CBC(iv0),
371 67 425.0 6.3 0.0 backend=default_backend())
372 67 3184.0 47.5 0.0 decryptor0 = cipher0.decryptor()
373
374 # instantiate actual HMAC-SHA256 of the ciphertext
375 134 1429.0 10.7 0.0 hmac0Act = hmac.HMAC(intKey, hashes.SHA256(),
376 67 386.0 5.8 0.0 backend=default_backend())
377
378 # decrypt ciphertext, until last block is reached
379 134 176.0 1.3 0.0 while fIn.tell() < inputLength - 32 - 1 - AESBlockSize:
380 # read data
381 134 214.0 1.6 0.0 cText = fIn.read(
382 134 121.0 0.9 0.0 min(
383 67 56.0 0.8 0.0 bufferSize,
384 67 70.0 1.0 0.0 inputLength - fIn.tell() - 32 - 1 - AESBlockSize
385 )
386 )
387 # update HMAC
388 67 916.0 13.7 0.0 hmac0Act.update(cText)
389 # decrypt data and write it to output file
390 67 1130.0 16.9 0.0 fOut.write(decryptor0.update(cText))
391
392 # last block reached, remove padding if needed
393
394 # read last block
395
396 # this is for empty files
397 67 59.0 0.9 0.0 if fIn.tell() != inputLength - 32 - 1:
398 67 65.0 1.0 0.0 cText = fIn.read(AESBlockSize)
399 67 63.0 0.9 0.0 if len(cText) < AESBlockSize:
400 raise ValueError("File is corrupted.")
401 else:
402 cText = bytes()
403
404 # update HMAC
405 67 308.0 4.6 0.0 hmac0Act.update(cText)
406
407 # read plaintext file size mod 16 lsb positions
408 67 63.0 0.9 0.0 fs16 = fIn.read(1)
409 67 54.0 0.8 0.0 if len(fs16) != 1:
410 raise ValueError("File is corrupted.")
411
412 # decrypt last block
413 67 1482.0 22.1 0.0 pText = decryptor0.update(cText) + decryptor0.finalize()
414
415 # remove padding
416 67 79.0 1.2 0.0 toremove = ((16 - fs16[0]) % 16)
417 67 58.0 0.9 0.0 if toremove != 0:
418 63 69.0 1.1 0.0 pText = pText[:-toremove]
419
420 # write decrypted data to output file
421 67 90.0 1.3 0.0 fOut.write(pText)
422
423 # read HMAC-SHA256 of the encrypted file
424 67 55.0 0.8 0.0 hmac0 = fIn.read(32)
425 67 61.0 0.9 0.0 if len(hmac0) != 32:
426 raise ValueError("File is corrupted.")
427
428 # HMAC check
429 67 623.0 9.3 0.0 if hmac0 != hmac0Act.finalize():
430 raise ValueError("Bad HMAC (file is corrupted).")
======================================
Total time: 18.7916 s
File: python/lib/python3.9/site-packages/pyAesCrypt/crypto.py
Function: stretch at line 59
Line # Hits Time Per Hit % Time Line Contents
==============================================================
59 @profile
60 def stretch(passw, iv1):
61
62 # hash the external iv and the password 8192 times
63 67 77.0 1.1 0.0 digest = iv1 + (16 * b"\x00")
64
65 548931 191790.0 0.3 1.0 for i in range(8192):
66 548864 9903957.0 18.0 52.7 passHash = hashes.Hash(hashes.SHA256(), backend=default_backend())
67 548864 2176946.0 4.0 11.6 passHash.update(digest)
68 548864 2363760.0 4.3 12.6 passHash.update(bytes(passw, "utf_16_le"))
69 548864 4155059.0 7.6 22.1 digest = passHash.finalize()
70
71 67 18.0 0.3 0.0 return digest
Hi, I've been trying to encrypt a string using this. The function works fine when called from python interpreter
or used in the main thread. However, the following doesn't work: when it is used inside a thread:
import io
import threading
import pyAesCrypt
def caller():
toencrypt = io.BytesIO(b"Hello World")
encrypted = io.BytesIO()
pyAesCrypt.encryptStream(toencrypt, encrypted, "pass1122", 64*1024) # Stuck here
print(encrypted.getvalue())
def main():
t = threading.Thread(target=caller)
t.daemon = True
t.start()
while True:
pass
if __name__ == "__main__":
main()
Thanks for working on this implementation.
I wanted to integrate pyAESCrypt with twisted and other async i/o and I ended up with this code
https://gist.github.com/adiroiban/b70f5257b10439c90aaa9dd6f6fda6ca
Let me know if you would like to have async support for this library.
I am happy to send a complete PR with automated tests.
I am not sure about the API...as it was a quick fix.
The structure should be the same...maybe some names should be changed.
For the AESCryptWriter ... we don't really new getHeader ... as it can return the header data together with the first cal to encryptChunk .... but I prefer to have a separate call for headers.
Also getFooter can be renamed to getFinal or finalize .. to have a similar API naming between crypt and decrypt.
Thanks again!
Hi,
I would love to get your feedback how we can make more difficult to decrypt encrypted file at hostile docker container ?
Do you know pyarmor ? How would be the best/bad sides of the both pyaescrypt and pyarmor
Currently, Travis-CI is configured to run tests on GNU/Linux only.
It would be nice (and useful, see #7) to have tests running on Windows and OS X too.
Can you make it, so it deals and encrypts strings instead of files ?
code is all working inside of pycharm however when running outside of it I get the no module found error despite latest version being installed and present on the system
I followed everything clearly on the python interpreter but it shows import error as
`
import pyAesCrypt as pc
pc.encryptFile("/Users/Sanju/Documents/views.py", "/Users/Sanju/Documents/views.aes", "password", 64 * 1024)
Traceback (most recent call last):
File "<pyshell#1>", line 1, in
pc.encryptFile("/Users/Sanju/Documents/views.py", "/Users/Sanju/Documents/views.aes", "password", 64 * 1024)
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyAesCrypt\crypto.py", line 97, in encryptFile
key = stretch(passw, iv1)
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyAesCrypt\crypto.py", line 65, in stretch
passHash = hashes.Hash(hashes.SHA256(), backend=default_backend())
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\hazmat\backends_init_.py", line 15, in default_backend
from cryptography.hazmat.backends.openssl.backend import backend
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\hazmat\backends\openssl_init_.py", line 7, in
from cryptography.hazmat.backends.openssl.backend import backend
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 16, in
from cryptography import utils, x509
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\x509_init_.py", line 8, in
from cryptography.x509.base import (
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\x509\base.py", line 16, in
from cryptography.x509.extensions import Extension, ExtensionType
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\x509\extensions.py", line 24, in
from cryptography.x509.general_name import GeneralName, IPAddress, OtherName
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cryptography\x509\general_name.py", line 10, in
from email.utils import parseaddr
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\email.py", line 1, in
import smtplib
File "C:\Users\Sanju\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", line 47, in
import email.utils
ModuleNotFoundError: No module named 'email.utils'; 'email' is not a package`
I noticed that an encrypted file gets a header like
AES� �CREATED_BY pyAesCrypt 0.4.3
and I found that it gets added in crypto.py line 165f
and I would like a way to make this optional.
I don't know if removing it would cause issues with decrypting later on but I don't
want anyone to be able to just see what tool was used for the encryption and then
google/search its weaknesses. If that is reasonable.
Running windows10 machine
Comparable to #6
decryptFile
function throws FilePermissionError
when wrong key is provided.
possible cause:
https://github.com/marcobellaccini/pyAesCrypt/blob/master/pyAesCrypt/crypto.py#L267
File is being removed while inside with
block which has the file open.
password="123456"
pyAesCrypt.encryptFile("data.txt", "output1.aes", password)
pyAesCrypt.encryptFile("data.txt", "output2.aes", password)
"output1.aes" and "output2.aes" will have different md5 values.
What I need is that "output1.aes" and "output2.aes" should be the same.
I have a directory with 4 text files now when I run try encrypt file function I am getting this below error
Traceback (most recent call last):
File "c:\Users\Documents\user\1\myfile.py", line 4, in <module>
encrypt_dir(r'D:\New folder')
File "C:\Python38\lib\site-packages\pycryptobox\encryption.py", line 49, in encrypt_dir
chck = pyAesCrypt.encryptFile(
File "C:\Python38\lib\site-packages\pyAesCrypt\crypto.py", line 95, in encryptFile
encryptStream(fIn, fOut, passw, bufferSize)
File "C:\Python38\lib\site-packages\pyAesCrypt\crypto.py", line 118, in encryptStream
if len(passw) > maxPassLen:
TypeError: object of type 'NoneType' has no len()
Can you kindly check & let me if there is any limit on using the password length ,currently I am storing pass in a env file MY_PASSWORD=mysecretpassword & loading inside main file load_dotenv , password = os.getnev('MY_PASSWORD")
also if I declare the password inside the script it seems to works
Just curious, can you make a encryption/decryption program and sell it (not releasing source)?
I know that you can make an open source one that is free.
Thanks for this module, Actually I have one encrypt video file and in my player I want to play this video, So can you tell me how can I decrypt the whole video into memory without storing it as a file?
In short I just wanted to know how to read the video file and then decrypt it into memory using pyAesCrypt
I think we can create .tmp file and rename/move it after success encryptFile/decryptFile process
I encrypted the file using pyAesCrypt.
I want to open and decrypt a file in Python3 and work with it further without saving the decrypted content to disk (work in RAM)
Example:
with open("data.txt.aes", "rb") as fIn:
encFileSize = stat("data.txt.aes").st_size
with open("dataout.txt", "wb") as fOut:# NO
fOut = [] # yes
pyAesCrypt.decryptStream(fIn, fOut, password, bufferSize, encFileSize)
print(fOut)
also would be great to
encFileSize = stat("data.txt.aes").st_size
maybe could be inferred automatically?
Thanks for this fantastic package. Is it possible to verify if a password is valid before decryption?
Environment
Description
The attached shell script will first create a /tmp/reprobug directory and subdirectories into which two versions of pyAesCrypt are installed. It will then create a test program and run it using v6.0.0 and v6.1.0 of the package. The test program will encrypt and decrypt its own source code and compare the result to the original source. With the old version of the package, the test passes. With v6.1.0, an exception halts program execution before the decryption occurs.
Expected behavior
Test should pass using any version of the package
How to Reproduce
Rename the attached file from reprobug.TXT to reprobug and run "zsh reprobug"
reprobug.TXT
Add folder .idea
to .gitignore
Sir, I want to stop encryption/decryption during process in my project. But I don't know how to do that please help. I don't care if my file get corrupted. I just want to stop encryption/decryption during its process..
Thank u!!!!!!
I have tarballs that are about 81M but grow to 2.8G when encrypted with pyAESCrypt encryptStream
.
What could be happening here?
Hi, I'm happily using pyAesCrypt on Mac OS X and want to install pyAesCrypt on Windows too.
When running below command it fails in the end because it cannot install cryptography (did not include all code, just showing that it collects cryptography). I went through readme and documentation for pyAesCrypt but did not see any requirements/dependencies mentioned.
Is cryptography a requirement or maybe some kind of misconfiguration on my Windows system?
c:\bitbucket.org\DataFitness\controloneagent\Tools\Python\PyPy\pypy3.7-v7.3.4-wi
n64>pypy3.exe -m pip install pyAesCrypt
Collecting pyAesCrypt
Downloading pyAesCrypt-6.0.0-py3-none-any.whl (15 kB)
Collecting cryptography
Downloading cryptography-3.4.7.tar.gz (546 kB)
|████████████████████████████████| 546 kB 3.3 MB/s
would i have to make a text file then convert it into a AES file then read it? also how do i set it to AES-256?
also, for my password, should i just slap my keyboard(not literally) and copying the text it makes as my new password?
First and foremost, thanks a lot for publishing this great library!
For my use case, I use the streaming functionality, processing files of relatively large sizes (MBs to GBs). I figured out that the current stream implementation immediately loads and decrypts the whole stream when calling decryptStream
. While this is fine for small files, it limits the library usage to files that fit into the available host memory. A more robust approach would be to load and decrypt the amount required by the read
function on demand.
This code block is responsible for an immediate decryption of the whole payload.
`def decryptStream(fIn, fOut, passw, bufferSize=bufferSizeDef, inputLength=None)`
[...]
# decrypt ciphertext, until last block is reached
last_block_reached = False
while not last_block_reached:
# read data
cText = fIn.read(bufferSize)
# end of buffer
if len(fIn.peek(32 + 1)) < 32 + 1:
last_block_reached = True
cText += fIn.read()
fs16 = cText[-32 - 1] # plaintext file size mod 16 lsb positions
hmac0 = cText[-32:]
cText = cText[: -32 - 1]
# update HMAC
hmac0Act.update(cText)
# decrypt data and write it to output file
pText = decryptor0.update(cText)
# remove padding
if last_block_reached:
toremove = (16 - fs16) % 16
if toremove:
pText = pText[:-toremove]
fOut.write(pText)
A clean solution to this would be to create a file-like reader class implementing the methods def read(self, size=-1)
, def tell(self)
and def close(self)
, which handles decryption on demand.
I created such an implementation for my project, and if you are interested, I am willing to create a pull request for integration.
Using your example from the readme:
import pyAesCrypt
from os import stat, remove
# encryption/decryption buffer size - 64K
bufferSize = 64 * 1024
password = "foopassword"
# encrypt
with open("data.txt", "rb") as fIn:
with open("data.txt.aes", "wb") as fOut:
pyAesCrypt.encryptStream(fIn, fOut, password, bufferSize)
# get encrypted file size
encFileSize = stat("data.txt.aes").st_size
# decrypt
with open("data.txt.aes", "rb") as fIn:
with open("dataout.txt", "wb") as fOut:
try:
# decrypt file stream
pyAesCrypt.decryptStream(fIn, fOut, password, bufferSize, encFileSize)
except ValueError:
# remove output file on error
remove("dataout.txt")
When I provide a false key during decryption the above produces a permission error (Running windows 10). Problem here is removing the file while it is still open (being inside the with
statement).
Below works as expected:
# decrypt
with open("data.txt.aes", "rb") as fIn:
try:
with open("dataout.txt", "wb") as fOut:
# decrypt file stream
pyAesCrypt.decryptStream(fIn, fOut, password, bufferSize, encFileSize)
except ValueError:
remove("dataout.txt")
Hi,with previous version of it i was able to play video files while decrypting a video file.I don't remember that version.Now I am facing problem after reinstalling python and pyAescrypt.
can u say which which version support that thing??
i am getting error,that is** "Moov Atom Not Found"**
I'm facing an issue decrypting the file after Pyinstaller.
The file /x/y/z/abc.pt.aes
Exists!
decrypting /x/y/z/abc.pt.aes to /tmp/_MEIyWW0kG/abc.pt
Traceback (most recent call last):
File "pyAesCrypt/crypto.py", line 258, in decryptFile
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/_MEIyWW0kG/abc.pt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "pyAesCrypt/crypto.py", line 271, in decryptFile
OSError: Unable to write output file.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "serve.py", line 63, in <module>
File "backend_code.py", line 50, in __init__
File "backend_code.py", line 123, in decrypt_files
File "decrypt.py", line 36, in decryptfile
File "pyAesCrypt/crypto.py", line 279, in decryptFile
OSError: File "/x/y/z/abc.pt.aes" was not found.
Any suggestions?
Feature request:
Be able to decrypt a part of the stream, for example part of a very large file.
According to https://stackoverflow.com/a/48833938 it should be possible by a stream.seek(position - 16) to set the IV, then decrypt as normal.
Thanks in advance!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.