Custom Style Generation about handwriting_line_generation HOT 4 CLOSED

Sorry it wasn't more obvious, but there is a much easier way to generate handwriting based on an example image. generate.py has an f option, which will ask for two image paths (to handwriting line images) and the target text and generate a series of images interpolating between the two styles it extracts. This doesn't save the style vector, but you could modify the code to if you wanted that. You may not want the interpolation, but the first and last images will be 100% the two respective styles. You could even give it the same image twice. If you want to extract the style from several images, you can just append them horizontally (that's how the model does it in training).

It is likely there are two primary reason for poor style matching:

1. The model doesn't actually mimic a partiular style very well. It mostly mimics global/general things like slant, but isn't good enough to recreate specific character shapes.
2. Your images are outside of the distribution images it was trained on (IAM). The IAM dataset was captured and preprocessed in a very uniform way. You probably will need to modify your images (contrast, cropping, ink thickness) to resemble the IAM images more closely.

from handwriting_line_generation.

Thank you for your time and for your quick reply
it really makes sense.

One more question regarding this topic:

I think I already fell through this problem, which was caused by both of the reasons that you've suggested, after training a model called GANWriting and trying to use it to generate my own handwriting.
In your great paper, I saw that you may be familiar with the GANWriting paper, as I noticed it was referenced there.

Then, I tried to solve reason number 2 by training shortly the GANWriting pre-trained (on IAM) model on my own handwriting samples. After trying to test the model and to generate my own handwriting given new samples that consist of my handwriting style and distribution, the results were as you mentioned in reason number 1: The model only mimicked global/general features like tilt and thickness and didn't focus on the low-level features as letter shapes, etc.

My question is:
Do you think that the same process should be useful, using your model in order to deal with reason number 2 (and not only modifying the images as contrast, cropping, ink thickness)? Does training your pre-trained checkpoint on my own handwriting could surpass the results I gained after I did so using the GANWriting model?

Thanks again!

from handwriting_line_generation.

Theoretically, you should be able to train/fine-tune any generation model on a single author's data and get good reconstruction of that author. The gotcha is you may not have enough data for a given author. Of course, you can always create more examples of your own handwriting. How much data is needed? I don't really know, but I'd imagine 20 lines covering all letters (lowercase and uppercase) multiple times would be enough.

I'm not sure why that wouldn't have worked with GANWriting, unless there weren't enough examples of your handwriting. Making the images look closer to the original data should make it so you need less data.

from handwriting_line_generation.

This is very helpful! Thanks for the motivation.
In continuation, I may try this and train the model on my own handwriting.
I am closing this issue as I have no more questions regarding this topic.

from handwriting_line_generation.