Comments (8)
@Qhuangcn: see my repo CTCDecoder.
from crnn.
@chengzhanzhan I have also observed the same phenomenon. Perhaps it's not a bug, but a strategy learned by LSTM. Notice that the CTC loss function does not care the position of individual letters, but only the final recognition results.
from crnn.
@bgshih I test 38k images for this phenomenon. Because of the convolution operation, a section of continuous pixels are represented by a shorter interval, which may results in the first position including some information of the first character.
Then I conduct experiments based on the
setting 1: 3 * 256 * 32 (channel * width * height) image ==> convolution layers ==> 128 * 65* 1 ==> LSTM layers ==> CTC. Here, each column (of 65) represent several columns of origin image.
setting 2: 3 * 256 * 32 (channel * width * height) image ==> convolution layers ==> 128 * 225* 1 ==> LSTM layers ==> CTC to make sure that each step (of 225) represents each column of the image.
A example is as follow:
setting 1 result:
3-----------22----6----22-----b----------------------------------
shows that the first position is character 3.
setting 2 result:
-----33------------------2----------------------6-------------------22-------------------------b---------------------------------------------------------------------------------------------------------------------------------
shows that the first several positions are blanks.
In CTC, each peak represents corresponding character, otherwise represent blank. Therefore, I think that the convolution strategy results in the phenomenon and CTC can index the probable position of each character.
from crnn.
@chengzhanzhan just want to let you know that I've observed the same phenomena with a different model, different framework (TensorFlow) and different input data (IAM). The last character is always predicted at the last time-step of the RNN output sequence. All other character-predictions are roughly aligned with the characters in the image. Look at the attached image, where the probability of the characters over time is plotted in the bottom-most graph: the "i", the "l"s and "t"s are aligned with the image while the final "e" is not.
from crnn.
@githubharald
Thankyou for your experimtent, do you figure out the reason caused this phenomena?
from crnn.
no.
from crnn.
@githubharald Hello, can you tell me how to calculate the character-level probability in ctc-beam search? Is there a tutorial of implementing that in tensorflow?
thank you
from crnn.
does anyone research on ctc of this phenomenon, for my model it shows the same problem as this. for example a blank image input to this, it sometime decode something instead of blank only output...... This is a serious problem on OCR application.
Let say I have 4 fields have to recognition, and 1 field is optional to fill in on the document, it should be able to recognition as blank image as blank output. if not the recognition accuracy within a document will be affected
from crnn.
Related Issues (20)
- compile fbthrift in ubuntu 16.04
- Could you give me some suggestions for training dataset? HOT 1
- About BK-tree you mentioned in the paper HOT 2
- The recognition results are reversed
- 我是cpu环境咋办
- create_dataset occur something wrong
- index-pack failed
- how to generate rgb lmdb by your tools
- Estimating receptive field to feature sequence vector mapping
- lmdb.Error:../data: Function not implemented
- Nan will appear with GPU training, but not with CPU
- lmdb.Error HOT 4
- how to run this command in google colab th main_train.lua ../models/foo_model/
- Update README as some links is invalid
- Data Requirement
- which dataset I can use for crnn training ? HOT 2
- Outdated docker
- How to convert Synth900k to lmdb? HOT 1
- manifest unknown -> manifest for kaixhin/cuda-torch:latest not found: manifest unknown: manifest unknown
- Creating lmdb dataset in google colab HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crnn.