This is a two stage lightweight and robust license plate recognition in MTCNN and LPRNet using Pytorch. MTCNN is a very well-known real-time detection model primarily designed for human face recognition. It is modified for license plate detection. LPRNet, another real-time end-to-end DNN, is utilized for the subsquent recognition. This network is attributed by its superior performance with low computational cost without preliminary character segmentation. The Spatial Transformer Layer is embeded in this work to allow a better characteristics for recognition. The recognition accuracy is up to 99% on CCPD base dataset with ~ 80 ms/image on Nivida Quadro P4000. Here is the illustration of the proposed pipeline:

The modified MTCNN structure is presented as below. Only proposal net (Pnet) and output net (Onet) are used in this work since it is found that skipping Rnet will not hurt the accuracy in this case. The Onet accepts 24(height) x 94(width) BGR image which is consistent with input for LPRNet.

LPRNet coding is heavily followed by sirius-ai's repo. One exception is that the spatial transformer layer is inserted to increase the accuracy reported on CCPD database as below:

• Download the CCPD data and put it into 'ccpd' folder

• run 'MTCNN/data_set/preprocess.py' to split training data and validation data and put in "ccpd_train" and "ccpd_val" folders respectively. python3 MTCNN/data_set/preprocess.py -image /home/zwzj/Downloads/CCPD2019/ccpd_base -dir_train ccpd/ccpd_train -dir_val ccpd/ccpd_val

• run 'MTCNN/data_preprocessing/gen_Pnet_val_data.py', 'MTCNN/data_preprocessing/gen_Onet_val_data.py','MTCNN/data_preprocessing/assemble_Pnet_val_imglist.py', 'MTCNN/data_preprocessing/assemble_Onet_val_imglist.py' for training data preparation (validation data part). cd MTCNN/data_preprocessing python3 gen_Pnet_val_data.py python3 assemble_Pnet_val_imglist.py python3 gen_Pnet_train_data.py python3 assemble_Pnet_train_imglist.py cd ../..

• run 'MTCNN/data_preprocessing/gen_Pnet_train_data.py', 'MTCNN/data_preprocessing/gen_Onet_train_data.py','MTCNN/data_preprocessing/assemble_Pnet_train_imglist.py', 'MTCNN/data_preprocessing/assemble_Onet_train_imglist.py' for training data preparation (train data part).

• run 'MTCNN/train/Train_Pnet.py' and 'MTCNN/train/Train_Onet.py cd MTCNN/train python3 Train_Pnet.py cd ../..

cd MTCNN/data_preprocessing python3 gen_Onet_val_data.py python3 assemble_Onet_val_imglist.py python3 gen_Onet_train_data.py python3 assemble_Onet_train_imglist.py cd ../..

cd MTCNN/train python3 Train_Onet.py cd ../..

mv MTCNN/train/pnet_Weights MTCNN/weights/ mv MTCNN/train/onet_Weights MTCNN/weights/

cd MTCNN python3 MTCNN.py --test_image ../test/5.jpg cd ..

• run 'LPRNet/data/preprocess.py' to prepare the dataset cd LPRNet/data python3 preprocess.py -image /home/zwzj/Downloads/CCPD2019/ccpd_base cd ../..

• run 'LPRNet/LPRNet_Train.py' for training python3 LPRNet_Train.py --batch_size 64

• run 'MTCNN/MTCNN.py' for license plate detection
• run 'LPRNet/LPRNet_Test.py' for license plate recognition
• run 'main.py' for both

• MTCNN
• LPRNet
• Spatial Transformer Layer
• LPRNet_Pytorch

Please give me a star if it is helpful for your research