I have seen another code about changing cnn feature to lstm as below
m = Permute((2, 1, 3), name='permute')(m) m = TimeDistributed(Flatten(), name='timedistrib')(m)
but yours is
# CNN to RNN inner = Reshape(target_shape=((32, 2048)), name='reshape')(inner) # (None, 32, 2048) inner = Dense(64, activation='relu', kernel_initializer='he_normal', name='dense1')(inner) # (None, 32, 64)
I think the reshape function is not correct, So could you tell if you have ever get good result?
thank you very much.

你好,你这个代码是只能识别长度为9的车牌吧?如果我想识别不固定长度的车牌,比如识别6,7,8这样长度的车牌时,代码应该怎么改?

I have a 25451 images in the train and 2000 images in the test set. The problem I am facing is the train and validation loss is high in every epochs. In my dataset a few images are of plates with two lines. So I would like to get a suggestion about the dataset size and it's nature. TIA

Hi,

I would like to know if we can dump input image slices corresponding to each time-step prediction? Say I use 32 time-steps. During inference, is it possible to get image-slices (of test-image) on which prediction was performed for each time-step?

Can your model deal with variable-length plate? In Image_Generator.py, you write Y_data[i]= text_to_labels(text), that mean "text" must have the length of 9. What about 7 or 8 characters? Or this model just work with 9-character plate? Thank you.

ValueError: cannot copy sequence with size 7 to array axis with dimension 9
Any help for this error?

ValueError: Cannot copy sequence with size 3 to array axis with dimension 19.

19 is max_text_len.

Any solution for this error?

Hi there,
I'm getting some problems when I try to run the code. I changed the data set to a same type of the original input but I can't pass this far...

Some one could help me?

RNN layer

lstm_1 = LSTM(256, return_sequences=True, kernel_initializer='he_normal', name='lstm1')(inner)  # (None, 32, 512)
lstm_1b = LSTM(256, return_sequences=True, go_backwards=True, kernel_initializer='he_normal', name='lstm1_b') (inner)
reversed_lstm_1b = Lambda(lambda inputTensor: K.reverse(inputTensor, axes=1)) (lstm_1b)

Exception has occurred: TypeError
Expected Operation, Variable, or Tensor, got 0
File "\CRNN-Keras-master\Model.py", line 62, in get_Model
lstm_1 = LSTM(256, return_sequences=True, kernel_initializer='he_normal', name='lstm1')(inner) # (None, 32, 512)
File "\CRNN-Keras-master\training.py", line 11, in
model = get_Model(training=True)

Is there any special purpose to implement custom decode function instead of using (more general) keras.backend.ctc_decode()?

. . .

Hi, thanks for your code above all.
And I am using the repo as an initial stage of a wild filed OCR task for unfixing length sequence.

How can I adjust the code to avoid the "invalid path problem of CTC"? I've searched some resources saying that there is a constraint between input length and label length but I was still confused the meaning.

Since my training set's maximum length string is over 35 words, and I just wonder if my padding will influence the performance of the task and how to really achieve the variant length sequence work properly, thanks

What is the accuracy that you got on multi line number plate only?

I have two question:

• The first, I have some real traffic images that I want to use for training model. But when I crop it to get only the plate, it maybe small ( ~ 50px). What size of the input images should be? And does it effect significantly to the accuracy of the model after training?
• The second, I want to train model for recognize both two and one layer plate. So how many image of each class should be prepared for a accuracy model?
  Thanks in advance!

True label =400
Predicted = 400lr

When I do prediction though results get correctly some unwanted text are always appended.
What can be the reason ? Accuracy is also show as 98%

img = cv2.resize(img,(self.img_w, self.img_h),None)
cv2.error: OpenCV(3.4.4) /io/opencv/modules/imgproc/src/resize.cpp:3784: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

I keep getting the above error inspite of resizing my images to 128x64 and I am not able to train the model because of this.

你好,请问你训练了大概几次?

Has anyone tried to replace default simple CNN structure with pretrained architectures like VGG, InceptionV3, etc.? On one hand the typical text image (like plates) is different in nature than typical photo (of a dog for example), but on the other hand maybe it would still be beneficial to use (first few) layers?

Hi,

I am new to neural network, so my question might be dumb.

After setting the correct environment and imported the correct library, I started to run the training.py file, but I constantly get a error:

ValueError: validation_steps=None is only valid for a generator based on the keras.utils.Sequence class. Please specify validation_steps or use the keras.utils.Sequence class.

And here is the whole output in the console:

Using TensorFlow backend.
2019-04-22 19:07:01.124992: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
...New weight data...
4 Image Loading start...
True
4 Image Loading finish...
5 Image Loading start...
True
5 Image Loading finish...
Traceback (most recent call last):
File "C:/Users/A02295947/Documents/Pycharm Projects/CRNN-Keras-master/training.py", line 41, in
validation_steps=int(tiger_val.n / val_batch_size))
File "C:\Users\A02295947\AppData\Local\Continuum\anaconda2\envs\tensorflow\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\A02295947\AppData\Local\Continuum\anaconda2\envs\tensorflow\lib\site-packages\keras\engine\training.py", line 2053, in fit_generator
raise ValueError('validation_steps=None is only valid for a'
ValueError: validation_steps=None is only valid for a generator based on the keras.utils.Sequence class. Please specify validation_steps or use the keras.utils.Sequence class.
Process finished with exit code 1

With mu friends' help, the conclusion was the training pictures are not enough. But I looked at your code, the "training.py" would ran the "Image_Generator.py" inside, so I guess the program would automatically generate images for the users? Otherwise what should I do to generate enough amount of images as I need?

In addition, what database did you use for massive actual Korean license plates?

Thanks,

-Yili

Hi, @qjadud1994
Q1- what's downsample_factor? if i increase the input_size of network time 2, that's 256*128, is necessary to change this parameter? and also be difference length input sequence of my own anpr.
Q2 - In your opinion, if i use other cnn, can i get best result? or this vgg is good?

HELLO! can you provide weights it takse alot of time in training and preparing dataset

Would you please show me an examle of txt or cvs files structure and naming convention of that files for training . thank you

crnn can recognize line of charactors by my understading, but images of yours appears to be 2 lines.
is my understand right?

I want to train the code, how can I prepare the data?

I have training set of size 26451 and test set of size 1000. I am running the code in colab. But when executing the trainin.py this error is shown.

New weight data
26451 images are being loaded ...
True
26451 images has been loaded.
1000 images are being loaded ...
True
1000 images has been loaded.
Epoch 1/30
  1/207 [..............................] - ETA: 31:57 - loss: 108.4210
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-15-cd1b0bd3606a> in <module>()
     90                     validation_steps = ceil( 
     91 
---> 92                         tiger_val.n / val_batch_size ) 
     93 
     94                    )

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    179             batch_index = 0
    180             while steps_done < steps_per_epoch:
--> 181                 generator_output = next(output_generator)
    182 
    183                 if not hasattr(generator_output, '__len__'):

StopIteration: 

Does anyone know what could be the problem?

Hi @qjadud1994 great work! I also need to train it for license plate recognition, can you give more info on how to go about it?

Hi,
Is it possible to increase the input size of network from 128x64 to 256x128 ?

Hi,
in the line

out_best = list(np.argmax(out[0, 2:], axis=1)) # get max index -> len = 32

why you used

2:0

in the out varibale.

Hi please let me know how to freeze vgg (or recommended layer for fine-tuning) layer for fine-tuning . Thank you

Has anyone succeed in modyfing standard keras_preprocessing.image.ImageDataGenerator class for this purpose? The TextImageGenerator() is fine to start, but if you want to utilize preprocessing functions it would be beneficial to use ImageDataGenerator?

I run trianing.py, occur this error

the detail information is:

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: slice index 0 of dimension 0 out of bounds.
[[node ctc/scan/strided_slice (defined at \Users\QZK\anaconda3\envs\crnn_keras\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
[[ctc/ToInt32_2/_503]]
(1) Invalid argument: slice index 0 of dimension 0 out of bounds.
[[node ctc/scan/strided_slice (defined at \Users\QZK\anaconda3\envs\crnn_keras\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.

Original stack trace for 'ctc/scan/strided_slice':
File "/ML_Projects/CRNN-Keras-master-new/training.py", line 16, in
model = get_Model(training=True)
File "\ML_Projects\CRNN-Keras-master-new\Model.py", line 86, in get_Model
loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([y_pred, labels, input_length, label_length]) #(None, 1)
File "\Users\QZK\anaconda3\envs\crnn_keras\lib\site-packages\keras\engine\topology.py", line 617, in call
output = self.call(inputs, **kwargs)
File "\Users\QZK\anaconda3\envs\crnn_keras\lib\site-packages\keras\layers\core.py", line 663, in call
return self.function(inputs, **arguments)
File "\ML_Projects\CRNN-Keras-master-new\Model.py", line 17, in ctc_lambda_func
return K.ctc_batch_cost(labels, y_pred, input_length, label_length)
File "\Users\QZK\anaconda3\envs\crnn_keras\lib\site-packages\keras\backend\tensorflow_backend.py", line 3941, in ctc_batch_cost
sparse_labels = tf.to_int32(ctc_label_dense_to_sparse(y_true, label_length))

The original paper seems not to have this slicing trick.

안녕하세요. 한국 분이신것 같아 한글로 적겠습니다.ㅎ

제가 올려주신 코드를 받아서 Text generator로 5개 타입에 대해 5000장 이미지를 트레이닝하고 1000장 이미지에 대해 원본 코드로 prediction 해보았는데요.

수행방법은 github에 있는 그대로 다른 부분은 수정하지 않았고 30번째 epochs의 결과값인 LSTM+BN5--30--17.991.hdf5를 Final_weight.hdf5 로 이름을 수정해서 prediction을 돌려봤더니 결과가 아래와 같습니다.
올려주신 결과와는 많이 상이한데 제가 돌리면서 누락한 부분이 있을까요? 문제점을 아직 찾지 못해서 한 번 봐주시면 감사하겠습니다.
Predicted: 01라18 / True: 서울 08아0188
Predicted: 1sk61 / True: 서울 14나3841
Predicted: 4ek15 / True: 서울 15수2594
Predicted: 광주 1eh82 / True: 서울 17모8052
Predicted: 광주 1fj49 / True: 서울 18허4839
Predicted: 전남 12마30 / True: 서울 41아0634
Predicted: 4ek15 / True: 서울 58모8149
Predicted: 4ek15 / True: 서울 62나6864
Predicted: 05다15 / True: 서울 64두9311
Predicted: 대전 5en19 / True: 서울 80두1049
Predicted: 4ek15 / True: 서울 80허8591

Predicted: 01라18 / True: 서울 08아0188
Predicted: 1sk61 / True: 서울 14나3841
Predicted: 4ek15 / True: 서울 15수2594
Predicted: 광주 1eh82 / True: 서울 17모8052
Predicted: 광주 1fj49 / True: 서울 18허4839
Predicted: 전남 12마30 / True: 서울 41아0634
Predicted: 4ek15 / True: 서울 58모8149
Predicted: 4ek15 / True: 서울 62나6864
Predicted: 05다15 / True: 서울 64두9311
Predicted: 대전 5en19 / True: 서울 80두1049
Predicted: 4ek15 / True: 서울 80허8591
Predicted: 67아02 / True: 서울 96거7915

Are you want to implement a bidirectional LSTM in the Model.py file between line 62 to 64? If the answer is YES. Here are a mistake of the Bi-LTSM's implementation.Did you forget to reverse the last_1b's output before input the add operation?

Hi @qjadud1994
I trained your Plates with your setting training but the loss is't reach that you have achieve 0.01 , but my val loss was 26 , why ?

OS : Ubuntu 16.04 LTS
GTX : 1080 ti
RAM :32 GB

Hi,
Please give me more information about training and prepare dataset, how do apply labels for this ?
please put this walk-through steps.
thanks.

Hi
I want to calculate the accuracy of the model during training, what do I do ? in the model.compile, there is a metrics option that can be 'accuracy', but I don't know, this accuracy metric is right for this problem or just only is for classification problem.
In your opinion, for this problem, I just write custom function accuracy?

Hi, You are did a great job and model. I liked very much. I have 2 issues . see the blow and help me out

1. while run trainin.py job i get below error

Traceback (most recent call last):
File "training.py", line 41, in
validation_steps=int(tiger_val.n / val_batch_size))
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 68, in fit_generator
raise ValueError('validation_steps=None is only valid for a'
ValueError: validation_steps=None is only valid for a generator based on the keras.utils.Sequence class. Please specify validation_steps or use the keras.utils.Sequence class.

#2 ) if I change validation_steps=int(tiger_val.n / val_batch_size)) to validation_steps=20 , I get below error. Kindly suggest me why I am getting these errors.

Epoch 1/30
Traceback (most recent call last):
File "training.py", line 41, in
validation_steps=5)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 251, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\callbacks.py", line 79, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\callbacks.py", line 338, in on_epoch_end
self.progbar.update(self.seen, self.log_values)
AttributeError: 'ProgbarLogger' object has no attribute 'log_values'

Hi, you have done a great job by the way. I am trying to understand implementation of the model.
I have a question regarding a line number 55 in Image_Generator.py.

input_length = np.ones((self.batch_size, 1)) * (self.img_w // self.downsample_factor - 2)

Am I right that img_w is downsampled by downsampling_factor due to size and number of maxpooling is applied? What I don't also get is why you substract 2 from it again?

When I load 43 images to train and 33 to test, with the command python training.py

This is the output:

(crnn-keras) C:\Users\X\Desktop\CRNN-Keras-master\CRNN-Keras-master>python training.py
Using TensorFlow backend.
2020-06-01 00:52:20.748876: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
...New weight data...
33  Image Loading start...
True
33  Image Loading finish...
43  Image Loading start...
True
43  Image Loading finish...
Epoch 1/30
Traceback (most recent call last):
  File "training.py", line 41, in <module>
    validation_steps=int(tiger_val.n / val_batch_size))
  File "C:\Users\X\Anaconda3\envs\crnn-keras\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\X\Anaconda3\envs\crnn-keras\lib\site-packages\keras\engine\training.py", line 2213, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "C:\Users\X\Anaconda3\envs\crnn-keras\lib\site-packages\keras\callbacks.py", line 76, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "C:\Users\X\Anaconda3\envs\crnn-keras\lib\site-packages\keras\callbacks.py", line 401, in on_epoch_end
    filepath = self.filepath.format(epoch=epoch + 1, **logs)
KeyError: 'val_loss'

Why it happends?

Only the fifth and sixth convolution layers are followed by the BN layer.

你好,请问test文件夹大概放几张测试图片?

Please clarify below information

1. Do i have to give any header to file?
2. Is this format to upload file is proper?
3. What should be the file name? Do i have to put file Data/train folder?
4. Do i need to change code of training.py to train with csv file?

Please help me with above question I am new to CRNN trying to learn new thing

I modified the Image_Generator a little and tried on my dataset of different length input, but I got the error message 'Not enough time for target transition sequence (required: 24, available: 23)'. Can you give some suggestions about it. Thank you.

Hi,
I've trained the VGG-GRU network for around 700k artificially generated images, and i got 96% accuracy for test data (about 140k generated image), but when i test the trained the model on real-images , the results are very bad, why ?

The code uses VGG architecture for feature extraction. My doubt is that, is it possible to begin training from VGG weights?

Hi,

I don't get it why in training.py there's a line 7, which forces the learning_phase to 0 (=test)?
K.set_learning_phase(0)

As far as I know for example BatchNormalization() layer behaves differently in 0 and 1 phase.