您好，请问您使用CCPD_BASE中的155k张图片训练的吗？因为我看到您的测试集的数量为45k？此外请问您有数据怎么增强操作吗，我训练之后的结果和您的相差很大。还有一个问题是我用您的权重进行测试，得到的精度非常低，这是为什么呢？

how i train this with my own dataset form scratch

There are many bugs in your code. We have to solve them one by one .tks

加载自己训练的stn权重发现车牌并没有矫正，加载作者原有的权重车牌得到有效矫正，有大佬遇到这种情况吗？

Thanks for your nice work!

I've noticed that the LPRNet is initialized using a pre-trained weights before training on CCPD. Here is the problem:

When I tried to train my model (another model) from scratch on CCPD, the ctc_loss got down to <=0.01 while accuracy remained <=1%.

I wonder is it necessary to apply pre-training on high quality dataset before CCPD?

Hello @xuexingyu24 ,

I followed your guide to train mtcnn and lprnet and got results as follows:

1. mtcnn: pnet and onet all got very low loss and high accuracy (ex. 99.xx%), but the testing result shows there are always some incorrect bounding boxes on a testing image. but, when I changed back the pnet and onet weights you provided, the result looks great. what am i missing?

mine:

yours:

2. lprnet: the final accuracy is slightly less than 96%. should it be higher?

ps. dataset that i used is ccpd_base, ccpd_db, ccpd_fn, ccpd_rotate, ccpd_tilt and ccpd_weather together.

thank you for your help in advance.

请问作者有研究下面两行代码不用pytorch的实现吗，比如用C/C++，python的实现？
grid = F.affine_grid(theta, x.size())
x = F.grid_sample(x, grid)

Can i use your model STN and LPRNet as init model. Thank you so much for awsome repo.

1）What should the filename of the dataset for training the LPRNet network look like?
2）How to solve the problem of Chinese garbled characters in the image names saved in the LPRNet/data/preprocess.py processing dataset

加载自己训练的stn权重发现车牌并没有矫正，加载作者原有的，车牌得到了有效矫正，有大佬遇到这种情况吗？

在Onet中添加landmark模块，训练时landmark loss不收敛，一直在0.0024附件上下波动，预测的时候也只是四个点预测不准，有做过的大佬可以交流一下吗？

您好，我看到您在LPRNet中插入了空間轉換層以提高識別精度。我想知道這個空間轉換層是如何工作的，以及它如何提高識別精度？此外，我也想了解一下在不使用空間轉換層的情況下，LPRNet的識別精度會是如何？謝謝。

为什么我训练出来的权重，在用于测试的时候，什么都没有显示出来，在训练的时候验证集准确率却很高

大佬，STN模型转onnx貌似转不了

I highly recommend adding a license to your repository. If it should be permissive, I suggest MIT or BSD-3-clause. GitHub recommends checking out this website: http://choosealicense.com

如鲁AA1233变成 鲁A1233 ,目前输出是18长度,需要改大还是调空白符的阈值

MTCNN中Train_Pnet.py的train_path = '../data_preprocessing/anno_store/imglist_anno_12.txt' 该文件是在哪个.py中生成的

Hi:
tks for your nice work!
Only support blue plate now?

OSError: cannot open resource when testing.

Hi, I tested your sample photo work, but it got error with other US license plates.
Do I need to retain CNN model or resize the photo?
Thanks,

IndexError Traceback (most recent call last)
in
13 STN.eval()
14
---> 15 bbox = bboxes[0, :4]
16 x1, y1, x2, y2 = [int(bbox[j]) for j in range(4)]
17 w = int(x2 - x1 + 1.0)

IndexError: index 0 is out of bounds for axis 0 with size 0

Hi I am running this block of codes but the shapes I am getting for bboxes1 and bboxes2 are different. Is there a reason to this and how can I correct it?

image1 = cv2.imread('test/6.jpg')
image2 = cv2.imread('train_data/train/class1/CB6692S.jpg')

input1 = np.copy(image1)
input2 = np.copy(image2)

bboxes1 = create_mtcnn_net(input1, (50, 15), device, p_model_path='MTCNN/weights/pnet_Weights', 
          o_model_path='MTCNN/weights/onet_Weights')
bboxes2 = create_mtcnn_net(input2, (50, 15), device, p_model_path='MTCNN/weights/pnet_Weights', 
          o_model_path='MTCNN/weights/onet_Weights')

bboxes1.shape (1,5)
bboxes2.shape (0,5)

https://github.com/pcb9382/PlateRecognition

TypeError: _resolve_type_from_object(): incompatible function arguments. The following argument types are supported:
1. (arg0: object, arg1: torch._C._jit_tree_views.SourceRange, arg2: Callable[[str], function]) -> torch._C.Type

Invoked with: typing.Union[int, NoneType], None, <function try_ann_to_type..fake_rcb at 0x000001E85E43CF28>

Hello , thank you for the great repository , i have one question , i want to train on custom license plates, can you tell me how to properly label the images , i am having a hard time understanding .
Thank you so much

Excuse me,how to get STN weights and what data set does it use

https://github.com/pcb9382/PlateRecognition

graph(%input.1 : Float(1, 3, 24, 94),
%backbone.0.weight : Float(64, 3, 3, 3),
%backbone.0.bias : Float(64),
%backbone.1.weight : Float(64),
%backbone.1.bias : Float(64),
%backbone.1.running_mean : Float(64),
%backbone.1.running_var : Float(64),
%backbone.4.block.0.weight : Float(32, 64, 1, 1),
%backbone.4.block.0.bias : Float(32),
%backbone.4.block.2.weight : Float(32, 32, 3, 1),
%backbone.4.block.2.bias : Float(32),
%backbone.4.block.4.weight : Float(32, 32, 1, 3),
%backbone.4.block.4.bias : Float(32),
%backbone.4.block.6.weight : Float(128, 32, 1, 1),
%backbone.4.block.6.bias : Float(128),
%backbone.5.weight : Float(128),
%backbone.5.bias : Float(128),
%backbone.5.running_mean : Float(128),
%backbone.5.running_var : Float(128),
%backbone.8.block.0.weight : Float(64, 64, 1, 1),
%backbone.8.block.0.bias : Float(64),
%backbone.8.block.2.weight : Float(64, 64, 3, 1),
%backbone.8.block.2.bias : Float(64),
%backbone.8.block.4.weight : Float(64, 64, 1, 3),
%backbone.8.block.4.bias : Float(64),
%backbone.8.block.6.weight : Float(256, 64, 1, 1),
%backbone.8.block.6.bias : Float(256),
%backbone.9.weight : Float(256),
%backbone.9.bias : Float(256),
%backbone.9.running_mean : Float(256),
%backbone.9.running_var : Float(256),
%backbone.11.block.0.weight : Float(64, 256, 1, 1),
%backbone.11.block.0.bias : Float(64),
%backbone.11.block.2.weight : Float(64, 64, 3, 1),
%backbone.11.block.2.bias : Float(64),
%backbone.11.block.4.weight : Float(64, 64, 1, 3),
%backbone.11.block.4.bias : Float(64),
%backbone.11.block.6.weight : Float(256, 64, 1, 1),
%backbone.11.block.6.bias : Float(256),
%backbone.12.weight : Float(256),
%backbone.12.bias : Float(256),
%backbone.12.running_mean : Float(256),
%backbone.12.running_var : Float(256),
%backbone.16.weight : Float(256, 64, 1, 4),
%backbone.16.bias : Float(256),
%backbone.17.weight : Float(256),
%backbone.17.bias : Float(256),
%backbone.17.running_mean : Float(256),
%backbone.17.running_var : Float(256),
%backbone.20.weight : Float(68, 256, 13, 1),
%backbone.20.bias : Float(68),
%backbone.21.weight : Float(68),
%backbone.21.bias : Float(68),
%backbone.21.running_mean : Float(68),
%backbone.21.running_var : Float(68),
%container.0.weight : Float(68, 516, 1, 1),
%container.0.bias : Float(68)):

Original python traceback for operator 14 in network torch-jit-export_predict in exception above (most recent call last):
Traceback (most recent call last):
File "to_onnx_lpr.py", line 32, in
outputs = rep.run(np.random.randn(1, 3, 24, 94).astype(np.float32))
File "/usr/local/lib/python3.6/dist-packages/caffe2/python/onnx/backend_rep.py", line 57, in run
self.workspace.RunNet(self.predict_net.name)
File "/usr/local/lib/python3.6/dist-packages/caffe2/python/onnx/workspace.py", line 63, in f
return getattr(workspace, attr)(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/caffe2/python/workspace.py", line 255, in RunNet
StringifyNetName(name), num_iter, allow_fail,
File "/usr/local/lib/python3.6/dist-packages/caffe2/python/workspace.py", line 216, in CallWithExceptionIntercept
return func(args, **kwargs)
RuntimeError: [enforce fail at conv_op_impl.h:38] C == filter.dim32(1) * G. 128 vs 64. Convolution op: input channels does not match: # of input channels 128 is not equal to kernel channels * group: 641
Error from operator:
input: "76" input: "backbone.8.block.0.weight" input: "backbone.8.block.0.bias" output: "77" name: "Conv_14" type: "Conv" arg { name: "strides" ints: 1 ints: 1 } arg { name: "pads" ints: 0 ints: 0 ints: 0 ints: 0 } arg { name: "dilations" ints: 1 ints: 1 } arg { name: "kernels" ints: 1 ints: 1 } arg { name: "group" i: 1 } device_option { device_type: 0 device_id: 0 }

I tried to change the data format, but still liked this.
Hope to get your reply, thank you!

Is there any reference to someone's knowledge which explains how to compute the accuracy of the LPRNet model output? I'm not interested into computing WER or CER, but an overall accuracy value for the LPRNet output

In my evaluation, the LPRNet weights are fine from pre-trained weights.

lprnet.load_state_dict(torch.load('LPRNet/weights/Final_LPRNet_model.pth', map_location=lambda storage, loc: storage))

STN.load_state_dict(torch.load('LPRNet/weights/Final_STN_model.pth', map_location=lambda storage, loc: storage)

However, the trained weights in checkpoint folder cannot be used.

lprnet.load_state_dict(torch.load(os.getcwd() + '/saving_ckpt/lprnet_Iter_001800_model.ckpt', map_location=lambda storage, loc: storage))

STN.load_state_dict(torch.load(os.getcwd() + '/saving_ckpt/stn_Iter_001800_model.ckpt', map_location=lambda storage, loc: storage))

The following main.py is modified for my environment, so the line numbers in the following error messages are not identical to main.py.

# ls saving_ckpt/
lprnet_Iter_000900_model.ckpt  stn_Iter_000900_model.ckpt
lprnet_Iter_001800_model.ckpt  stn_Iter_001800_model.ckpt

# python3 main.py -image /data/pics/
Traceback (most recent call last):
  File "main.py", line 29, in <module>
    lprnet.load_state_dict(torch.load(os.getcwd() + '/saving_ckpt/lprnet_Iter_001800_model.ckpt', map_location=lambda storage, loc: storage))
  File "/root/anaconda3/envs/eval/lib/python3.7/site-packages/torch/nn/modules/module.py", line 769, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for LPRNet:
	Missing key(s) in state_dict: "backbone.0.weight", "backbone.0.bias", "backbone.1.weight", "backbone.1.bias", "backbone.1.running_mean", "backbone.1.running_var", "backbone.4.block.0.weight", "backbone.4.block.0.bias", "backbone.4.block.2.weight", "backbone.4.block.2.bias", "backbone.4.block.4.weight", "backbone.4.block.4.bias", "backbone.4.block.6.weight", "backbone.4.block.6.bias", "backbone.5.weight", "backbone.5.bias", "backbone.5.running_mean", "backbone.5.running_var", "backbone.8.block.0.weight", "backbone.8.block.0.bias", "backbone.8.block.2.weight", "backbone.8.block.2.bias", "backbone.8.block.4.weight", "backbone.8.block.4.bias", "backbone.8.block.6.weight", "backbone.8.block.6.bias", "backbone.9.weight", "backbone.9.bias", "backbone.9.running_mean", "backbone.9.running_var", "backbone.11.block.0.weight", "backbone.11.block.0.bias", "backbone.11.block.2.weight", "backbone.11.block.2.bias", "backbone.11.block.4.weight", "backbone.11.block.4.bias", "backbone.11.block.6.weight", "backbone.11.block.6.bias", "backbone.12.weight", "backbone.12.bias", "backbone.12.running_mean", "backbone.12.running_var", "backbone.16.weight", "backbone.16.bias", "backbone.17.weight", "backbone.17.bias", "backbone.17.running_mean", "backbone.17.running_var", "backbone.20.weight", "backbone.20.bias", "backbone.21.weight", "backbone.21.bias", "backbone.21.running_mean", "backbone.21.running_var", "container.0.weight", "container.0.bias". 
	Unexpected key(s) in state_dict: "iters", "net_state_dict".

How to use my trained weights?

please help with annotation tool used for annotating on custom dataset

@xuexingyu24

Traceback (most recent call last):
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\preprocessing\get_Onet_train_data.py", line 61, in
bboxes = create_mtcnn_net(image, 50, device, p_model_path=None, r_model_path=None,o_model_path='../weights/onet_Weights')
TypeError: create_mtcnn_net() got an unexpected keyword argument 'r_model_path'

_当我把这个r_model_path删了之后，又会报出：

Traceback (most recent call last):
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\preprocessing\get_Onet_train_data.py", line 61, in
bboxes = create_mtcnn_net(image, 50, device, p_model_path=None,o_model_path='../weights/onet_Weights')
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\MTCNN.py", line 28, in create_mtcnn_net
bboxes = detect_onet(onet, image, bboxes, device)
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\MTCNN.py", line 123, in detect_onet
[dy, edy, dx, edx, y, ey, x, ex, w, h] = correct_bboxes(bboxes, width, height)
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\preprocessing..\util\util.py", line 138, in correct_bboxes
x1, y1, x2, y2 = [bboxes[:, i] for i in range(4)]
File "D:\火狐下载\License_Plate_Detection_Pytorch-master\License_Plate_Detection_Pytorch-master\mtcnn\preprocessing..\util\util.py", line 138, in
x1, y1, x2, y2 = [bboxes[:, i] for i in range(4)]
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

请问该怎么解决呢？

Just a small bug, there is no r_model_path in create_mtcnn_net. Therefore, get_Onet_train_data.py does not have to assign create_mtcnn_net with r_model_path anymore.

As the project says, the accuracy of the model works well. So I want to know if there is a paper in the project, and I can study it in more depth. I would be very grateful if you could provide that.

can you please share your dataset structure ?
thank you

Is it possible to use different sized box for STN? It gives an exception when changing the "im = cv2.resize(img_box, (94, 24), interpolation=cv2.INTER_CUBIC)" dimensions from (94,24) to any other size i.e. (94,25).

Exception given:
Exception has occurred: RuntimeError
Sizes of tensors must match except in dimension 2. Got 5 and 4

hello i'm running your code on windows and i'm using it for non English-language license plates(it is not Chinese either) and i managed to work around the fact that python couldn't read the file names(labels) of the images but now i keep getting this error:
img, label, length = sample
TypeError: 'NoneType' object is not iterable
the problem seems to be that the collate_fn function is getting none for input and therefore resulting in sample to be none too.

Hi,
I want to filter the output of the LPRNET to just show the plates that have a recognition probability of more than 60%.
How can I obtain the probability of any plate OCR recognition?

Thanks

gen_Onet_train_data.py文件中 63行的bboxes = create_mtcnn_net( r_model_path=None)这个方法里面该参数并没有存在 ，请问需要修改哪个地方呢？

i tried to train my model on my plate data.

"loss: 0.0000, train_accuracy: 0.1875, time: 0.28 s/iter, learning rate: 0.0001"

loss so small, but train_accuracy always <= 0.4 . I trained 5hour and can't get better result. How can i fix that ?

Is that OK if I don't split dataset into val & tra parts while training?

Is there any effect on training process?

In the Training on MTCNN, we should run
'MTCNN/data_preprocessing/gen_Onet_train_data.py'.

However, this file does not exist.

I want to output the landmark, I mean those points. I notice that the class Onet in MTCNN_net.py has a return c which was commented. Does this c have any thing to do with the landmark? Thanks!

https://github.com/pcb9382/PlateRecognition

您好，我能问个问题吗，在训练数据集的时候，出现下面的情况是什么问题呢？
RuntimeError: Expected tensor for argument #2 'targets' to have scalar type Int; but got torch.FloatTensor instead (while checking arguments for ctc_loss_cpu)
@xuexingyu24