Is it necessary to restart the program after processing one image?
How to run it continuously?

Now I'm playing around with Stefann a bit. Your readme was really helpful here! But I guess there's more knobs and dials to it than I expected. Was still wondering is there e.g. a way to make the word you're replacing longer? Also, your examples seem way more crips than my quick shot at it. Got any tips?

Original:

Edited

When I run the code, it is reporting the error integrity failure.

Hello, prasunroy. Thank you for the amazing work. I tried to train fannet by python stefann/fannet.py following your codes, but got ValueError: No gradients provided for any variable and it stopped training.
Could you give me a solution to fix the bug？

Hello, I noticed that in the stefann.py file, you have provided three methods for style transfer, transfer_color_pal, transfer_color_max, and the colornet implementation. I uncommented the lines from 480-487 in the stefann.py file to use the colornet model. When I tested the application on images from 'sample_images' folder using the colornet method with the provided pretrained weights, it did not work very well and produced blurry and inconsistent results. The results of colornet did not match the results given in the 'editing_examples' folder.

Given Result (Left - Original Image, Right - stefann generated image)

Output using colornet

There are many more cases where the colornet model is not performing as expected. Could you please help me with this?

I appreciate your excellent work on text editing. I tried to run FANnet with pretrained model on your datasets. So I downloaded the pretrained weights from here and datasets from here following README.

To generate results using the valid set as the input, I modified fannet.py and ran the following code

from skimage.metrics import structural_similarity as ssim
for data in valid_datagen.flow():
    [x, onehot], y = data
    out = fannet.predict([x, onehot]) 
    n = x.shape[0]
    for i in range(n):
        _x = x[i].reshape(64, 64)
        _gt = y[i].reshape(64, 64)
        _out = out[i].reshape(64, 64)
        _, _out_bin = cv2.threshold(_out,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
        sv = ssim(_gt, _out_bin, data_range=255, gaussian_weights=True, sigma=1.5, use_sample_covariance=False)
        print(sv)

But the SSIM value sv is far from what was claimed in the paper. I also tried to calculate the average SSIM w.r.t different source characters. There is also a large gap. So I am wondering if there exists some mistakes when running the model or just the SSIM calculation.

Hi,

Thanks for sharing your code. I am trying to reproduce your results, how many epochs does it take to converge to results similar to what you shared in your paper ?

Thanks

Thank you very much for your contribution in this field.
I still have some questions about the character replacement part. As you said in the paper, when training fannet and colornet, the input character size is 6464, but when performing character replacement in a real scene, some characters are much larger in size 6464 (for example, 128128), how did you solve this situation? Is it to directly resize the generated characters to 128128 or some other way?

After I run the command python stefann.py and open an image, what are the steps to replace the characters in that image?

Hi,

Thanks for sharing your awesome work! I am wondering if you have a pretrained model to edit numbers? Thanks.

I followed the installation instructions, but kept getting the following message.

[DEBUG] Loading application... integrity failure

There was no problem with my CUDA settings, and tensorflow was picking up my GPU just fine, so I tried taking stuff out of try/exceptions and found out that keras was failing on loading the models.

original_keras_version = f.attrs['keras_version'].decode('utf8')

AttributeError: 'str' object has no attribute 'decode'

Googled it, and found that it works at h5py version 2.10.0.
I think it would be nice if h5py versions were also included in conda settings.

Thank you anyways for your awesome work.
I'll try it out now since my bug is fixed.

I want to detect the color of the text in the original image. Any possible ways?

Can anyone make a Colab or easy-to-run version please? 🙏

Hi,

I was trying out your project but I'm stuck installing dependencies. I tried the conda installation method but it keeps failing (tried cpu, gpu and osx yamls).
[DEBUG] Loading application... integrity failure

I also think the yaml file here

I guess it should look more like the OSX version with a double == and tensorflow below pip.

It could be related to this issue: tensorflow/tensorflow#37316
but when I tried tensorflow>=2.1.0 it also failed.

Any idea how to solve this? Or is there an alternative way to install it? A docker container maybe?

Hello, after reading your paper, I have some questions about the input of FANnet.
In the paper, one of the inputs to FANnet is "a one-hot encoding v of length 26 of the target character", where 26 represents 26 capital letters. Have you ever tried to change 26 to 62 (uppercase letters + lowercase letters + numbers), because the same font theoretically has some commonalities, so they should also be able to complete the font style transfer. I am interested in this question. Do you have any comments or suggestions?

@mauromazzei @prasunroy
What about languages like Arabic & Hebrew?

Hi, thanks for your hard work on this project. It's really cool!
I've seen issue #7 but I still have some doubts. I would like to try to replace english text with it's corresponding chinese translation, but how can I do so if characters are stored in jpg file named as ASCII numbers? Chinese it's not included in ASCII.
Another question regarding chinese is, do I also need to generate new images, one for each character, in the colornet directory?
Your help would be much appreciated!