cwig / start_follow_read Goto Github PK

I am trying to train the model, but I have only one GPU. Can it be trained on single GPU? If not what changes will be required?
Thanks.

Hi! I think you have a typo in the installing dependencies step
conda env create -f environment.yml instead of
conda env create -f environment.yaml

Please tell me how to solve the problem that the libraries under the pip command cannot be installed when configuring the environment?

Hi!
I read your paper and you have compiled lines from IAM dataset and then use it as full page. The results are great. Will the model run easily on single lines instead of full page ? What will be the process to that ?

Hi,

I am trying to use the pre-trained model to reproduce the result. However, an error was raised due to some weights are missing when loading LF pre-train file:
KeyError: 'missing keys in state_dict: "set(['cnn.convRelu6.batchnorm6.bias', 'cnn.convRelu4.batchnorm4.weight', 'cnn.convRelu2.batchnorm2.weight', 'cnn.convRelu2.batchnorm2.bias', 'cnn.convRelu4.batchnorm4.bias', 'cnn.convRelu6.batchnorm6.weight'])"'

Thank you.

Hi.

Could you please send me an e-mail. I'd like to discuss an opportunity with you. In 2017 I attended your presentation at BYU’s Family History Technology Workshop.

[email protected]
Michael Iams

@cwig @ctensmeyer @herobd
In case I only want to extract Text-lines without the hwr step, how can I do that?
Perhaps only output a Page xml file?

In environment.yaml I had to make two changes in order to get the code to run.

[- torch==3.10] --> [- torch==0.3.1]

and

[- torchvision=0.2.1] --> [- torchvision==0.2.1]

This was on Ubuntu 18.04 with conda version conda 4.5.4

Hi,

I am trying to reproduce the result of your model with only SOL and LF module by withdrawing HWR module on your model, we used the pre-train model that you provided.

We test the model with ICDAR dataset.
The performance of SOL is quite good, but LF has a little problem. Some of the testing samples, the result is good, just like below one. (Test-B-050003)

However, we found that some of the testing samples are weird, for example (Test-B-050006):

I would appreciate some advice if you cloud share some idea to test the model. Is the issue related to the model input_height?

Hi,

I started looking into the code for handwriting recognition. I am unable to download the data from the links could you please help me in getting the data.

Thanks

Hi,
Please share the pointers to create the following files related to language model:
fst_path: "../hwn5-comp-2017/data/comp_lm/data/graph/HCLG.fst"
mdl_path: "../hwn5-comp-2017/data/comp_lm/data/lang_test/basic.mdl"
words_path: "../hwn5-comp-2017/data/comp_lm/data/graph/words.txt"
phones_path: "../hwn5-comp-2017/data/comp_lm/data/lang_test/phones.txt"

I installed Pykaldi but not able to make out where to generate these files for my dataset.
Thanks in advance.

Firstly, thank you for your paper "Start, Follow, Read: Full-Page End-to-end Handwriting Recognition".

I am trying to get a better understanding of the training of the modified vgg11 model you used in your paper. To this end, I translated your vg11 found in vgg.py does into Keras.

inputShape = (height, width, depth)
inputs = Input(shape=inputShape)
# Conv Block 1
x = Conv2D(64, (3, 3), activation='relu', padding='same')(inputs)
x = MaxPooling2D(pool_size=(2, 2), strides=(2, 2))(x)

# Conv Block 2
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2), strides=(2, 2))(x)

# Conv Block 3
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2), strides=(2, 2))(x)

# Conv Block 4
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2, 2), strides=(2, 2))(x)

# Conv Block 5
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)

# FC layers
x = Flatten()(x)
x = Dense(4, activation='linear')(x)

model = Model(inputs, x)

I trained this model with 5000 test images of a handwritten script and corresponding bounding box for a word off center as a baby step to replicate your SOL target approach.

sgd = keras.optimizers.SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss=losses.mean_squared_error, optimizer=sgd)

To keep things simple to start, my training data only present a subset of SOL data plus width and height. This modified vgg11 only learns the average the SOL data rather than a model that can predict sol parameters based on an input image.

Here are some samples of my output with green being ground thruth and red being predicted value from the vgg11 model:

predictY = model.predict(testX)
print_sample(testX, testY, predictY)

I would appreciate some suggestions as to why my copy vgg11 can't seem to learn anything about the potential SOL words presented to it or some insight into how your modified vgg11 was trained.

Regards,
John Grabner

Does anyone found a link to the RIMES database?