luojilab / django-mirage-field Goto Github PK

View Code? Open in Web Editor NEW

92.0 15.0 15.0 80 KB

Django model field encrypt/decrypt your data, keep secret in database.

License: MIT License

Java 3.20% Python 96.80%

django python encrypt cryptography cipher encryption aes cbc

django-mirage-field's Introduction

django-mirage-field

NOTE!!!

The maintainer tcitry has resigned from luojilab, he won't contribute this project ever, the latest code please refer: tcitry/django-mirage-field.

Introduce

A Django model fields collection that encrypt your data when save to and decrypt when get from database. It keeps data always encrypted in database. Base on AES, it supports query method like get() and filter() in Django.

Mirage can also migrate data from origin column to encrypted column in database with a good performance.

Features

Use settings.SECRET_KEY as secret key default or anyelse which length >= 32
Support CharField, TextField, IntegerField, EmailField
Support Django ORM's get(), filter() query method
Support AES-256-ECB and AES-256-CBC(v1.2.0)
Support PostgreSQL and MySQL database
Support Django model field db_index and unique attributes

Installation

pip install django-mirage-field

Usage

from mirage import fields
class TestModel(models.Model):
    age = fields.EncryptedIntegerField()

obj = TestModel.objects.get(age=18)
obj.id          # 1
obj.age         # 18
type(obj.age)   # int

database=# select * from testmodel where id = 1;
         id          |           age
---------------------+--------------------------
 1 | -bYijegsEDrmS1s7ilnspA==

from mirage.crypto import Crypto
c = Crypto()                      # key is optional, default will use settings.SECRET_KEY
c.encrypt('some_address')               # -bYijegsEDrmS1s7ilnspA==
c.decrypt('-bYijegsEDrmS1s7ilnspA==')   # some_address

Settings

MIRAGE_SECRET_KEY
MIRAGE_CIPHER_MODE (v1.2.0+)
MIRAGE_CIPHER_IV (v1.2.0+)

MIRAGE_SECRET_KEY

You can use the settings.SECRET_KEY as default key, if you want custom another key for mirage, set the MIRAGE_SECRET_KEY in settings.

Mirage will get the settings.MIRAGE_SECRET_KEY first, if not set, mirage will get the settings.SECRET_KEY.

MIRAGE_CIPHER_MODE

MIRAGE_CIPHER_MODE is optional, choices are below, If don't set, default is ECB.

ECB
CBC

MIRAGE_CIPHER_IV

MIRAGE_CIPHER_IV is optional, if you don't set, it will use a default: "1234567890abcdef", it's length must be 16.

Model Fields

EncryptedTextField
EncryptedCharField
EncryptedEmailField
EncryptedIntegerField
EncryptedURLField(v1.3.0+)

Data Migrate

AddmiragetoINSTALLED_APPS

Way 1. Migrations

add app_name,model_name,field_name in migrations.RunPython

from mirage.tools import Migrator

migrations.RunPython(Migrator("app_name", "model_name", "field_name").encrypt, reverse_code=Migrator("app_name", 'model_name', 'field_name').decrypt),

Way 2. Commands

Options:

--app
--model
--field
--method (optional: encrypt, decrypt, encrypt_to, decrypt_to, copy_to)
--tofield (need when use encryt_to, decrypt_to, copy_to method)

Optional options:

--offset ("select * from xxx where id > offset")
--total ("select * from xxx order by id limit total")
--limit: set the query count in every update, default is 1000, if you set -1, mirage will query all rows one time to update.

Examples

./manage.py mirage --app=yourapp --model=testmodel --field=address --method=encrypt --offset=2000000 --total=3000000

./manage.py mirage --app=yourapp --model=testmodel --field=address --method=encrypt_to --tofield=encrypted_address

Exceptions

from mirage import exceptions

EncryptedFieldException

Performance

With ECB mode

Migrate data: 6000,000 columns takes 40 minutes, Average 1 column/2.5ms

Only encrypt/decrypt: Average 1 value/ms

Clients

Java

django-mirage-field's People

Contributors

Stargazers

Watchers

Forkers

irumiha ecrow liugaohui shadeimi sommelon barseghyanartur eberonsh ambientlighter zvibaratz naseemmdali esteban-alo tcitry toanalien

django-mirage-field's Issues

migrate error wheel pk is UUID not int

try try just know what bug

Is it possible to add encrypted URL fields?

Is it possible to add encrypted URL fields?
It would be useful in cases of external URL's

Bug: Migrator.decrypt() replaces null with an empty string

If a database column has null values they will be replaced with the empty strings '' after calling Migrator.decrypt. Found during development with Django and PostgreSQL.

EncryptedJSONField doest not decrypt automatically

Hi,

It seems that EncryptedJSONField does not automatically decrypts the value when fetched.

Can not search in Django Admin List

id          |           phone
---------------------+--------------------------
 123 | -bYijegsEDrmS1s7ilnspA==

obj = TestModel.objects.get(phone=18866677777)
obj.id          # 123
obj.phone       # 18866677777
type(obj.phone) # int

#admin.py
search_fields = ['phone']

In Django Admin, I can search by '-bYijegsEDrmS1s7ilnspA==', but I cannot search by '18866677777'.

Can't read SECRET_KEY set with django-configurations

Hi, I've tried to use django-mirage-fields and I use django-configurations for vars settings,
mirage can't read SECRET_KEY.

settings.py

from configurations import Configuration, values

class MyProject(Configuration):
    SECRET_KEY = values.SecretValue()

.env

DJANGO_SECRET_KEY=SecretSecretSecretSecretSecretSecret

values of SECRET_KEY returned by mirage is None

Is there a way to set another default KEY for mirage?

Cryptographic API Misuse Vulnerability

Description:

In the "django-mirage-field/mirage/crypto.py", I have identified security vulnerabilities about insecure cryptographic algorithm. ECB mode encryption is not semantically secure, which means it might cause information leakage.

Location:

https://github.com/luojilab/django-mirage-field/blob/master/mirage/crypto.py#L31

 encryptor = Cipher(algorithms.AES(self.key),
                           modes.ECB(), default_backend()).encryptor()

https://github.com/luojilab/django-mirage-field/blob/master/mirage/crypto.py#L39

 decryptor = Cipher(algorithms.AES(self.key),
                           modes.ECB(), default_backend()).decryptor()

Reference

CWE-327: Use of a Broken or Risky Cryptographic Algorithm

Recommendate

Use other cipher mode such as CFB or CTR with random IV.

ECB mode not recommended

Hi,

Thanks for the helpful implementation.
Is it possible to change ECB mode with CBC mode?
ECB mode is definitely not recommended because it's weak.

Best,
Severin

Is it possible to add encrypted JSON fields?

Hello, it's me again!!
Congratulations for the great work with this repository, it has become essential in several projects of my team.
Currently we are using the JSONField (models.JSONField), as it was done in the URLField is it possible to do the same with the JSON?
The reason is that in some external API returns there are data that I would like to have encrypted inside the database and with mirage it would be a more ideal way since that is what I have used in absolutely all cases.

Is this repository still maintained?

Hello,

The original maintainer @tcitry mentioned in this commit that he'll no longer be maintaining django-mirage-field. Could you clarify if this repository is still being maintained or not?

Thanks!

Migrator operations fail on an empty database

I have a fix for an issue we ran into:

Unable to install django-mirage-field in Windows machine

Facing the following error(UnicodeDecodeError) while installing django-mirage-field:

Migrator converts NULL values to a string "None"

Hi.
I am using SQLite 3 and I have this model

class AuthConfig(models.Model):
   username = fields.EncryptedCharField(max_length=150, null=True, blank=True, verbose_name=_("Username"))
   password = fields.EncryptedCharField(null=True, blank=True, max_length=150, verbose_name=_("Password"))
   token = fields.EncryptedCharField(max_length=1500, null=True, blank=True, verbose_name=_("Token"))

My migration looks like this

migrations.RunPython(
    Migrator("integrations", "AuthConfig", "username").encrypt, reverse_code=Migrator("integrations", 'AuthConfig', 'username').decrypt
        ),
migrations.RunPython(
    Migrator("integrations", "AuthConfig", "password").encrypt, reverse_code=Migrator("integrations", 'AuthConfig', 'password').decrypt
        ),
migrations.RunPython(
    Migrator("integrations", "AuthConfig", "token").encrypt, reverse_code=Migrator("integrations", 'AuthConfig', 'token').decrypt
),

When the original field value is saved in the database as NULL, applying the migration will change the value to a string "None" because of the string interpolation used to construct an UPDATE statement in Migrators execute() method.

To reproduce:

Make a model with an optional field (null=True)
Create a record with that field being NULL
Convert the field to an Encrypted field
Run the migration with Migrator

Strange issue with Python 3.7 and 3.8

Hi,

I'm developing a Django application, and even though everything works fine in my local dev environment (Python 3.9), my CI build for Python 3.7 and 3.8 is failing (surprisingly, Python 3.6 is passing, and so is 3.9).
The exception I'm getting looks like this:

  File "/root/pylabber/accounts/models/__init__.py", line 9, in <module>
    from accounts.models.export_destination import ExportDestination
  File "/root/pylabber/accounts/models/export_destination.py", line 15, in <module>
    class ExportDestination(TitleDescriptionModel):
  File "/root/pylabber/accounts/models/export_destination.py", line 29, in ExportDestination
    password = fields.EncryptedCharField(
  File "/root/pylabber/.tox/py38-django3/lib/python3.8/site-packages/mirage/fields.py", line 13, in __init__
    self.crypto = Crypto(key)
  File "/root/pylabber/.tox/py38-django3/lib/python3.8/site-packages/mirage/crypto.py", line 60, in __init__
    assert len(key) >= 32, "mirage key length must more than 32!"
AssertionError: mirage key length must more than 32!

The relevant field definition looks like:

class ExportDestination(TitleDescriptionModel):
    ...
    password = fields.EncryptedCharField(
        max_length=128, blank=False, null=False
    )

Any help figuring out the cause for this would be greatly appreciated.
Thanks for all your great work,
Zvi