davidhalladay / frido Goto Github PK

Thanks for your excellent work!
In the paper, you have mentioned that the model size of Frido-f16f8 is 697.8M, which include the parameters of Unet, MS-VQGAN, and Bert. When i use the code you relase and run "bash tools/eval_t2i.sh", I found that the size of Unet is 512M, and the size of VQ-Gan is 68M, and Bert is 186M, the sum of this three is not 697M, it seems that the 697M does not include VQ-gan.
Am I mistaken? Ask your advice

Thanks for the great code.

I actually got an error when I tried Sg2Iw

bash tools/frido/eval_sg2i_coco.sh

RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
    return self.collate_fn(data)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 128, in collate
    return elem_type({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 128, in <dictcomp>
    return elem_type({key: collate([d[key] for d in batch], collate_fn_map=collate_fn_map) for key in elem})
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 139, in collate
    raise RuntimeError('each element in list of batch should be of equal size')
RuntimeError: each element in list of batch should be of equal size

Is there any way to resolve this?

Hello,

Thank you for your amazing work.
I am confused by the BertEmbedder being used in the Layout2Image config when the conditioning key is the bbox coordinates and not captions. Can you explain how it comes to play. Thank you

Thanks for your great work!

SG2Im, Layout2Im, LostGAN, they train on the deprecated coco-stuff with 24,972 training images. Do you train on the deprecated coco-stuff 2017 segmentation challenge or the full coco-stuff 2017?

Thanks for your impressive work! I am very interested in layout-to-image generation.

1. For COCO-stuff, you sample 2048 images. Do you sample 5 times for each image just like sg2im ? 2048 x 1 or 2048 x 5?
2. For VG, 3096 x 1 ?
3. Have you tried LDM-4 using LDM's pretrained model in openimage?

Hi, congrats for the great work!
I was trying to reproduce the FID on coco for the layout2image task, but got and FID ~= 60.

This is what I'm running:

bash tools/download.sh
bash tools/eval_layout2i.sh
fidelity --gpu 0 --fid --input2 exp/layout2i/frido_f8f4/samples/v0/img/inputs --input1 exp/layout2i/frido_f8f4/samples/v0/im/sample

Am I doing something wrong?

p.s. I noticed you used 2,048 val images to compute the FID, do you think this can be the cause? (/can you share the file_ids of those images?). Nevertheless the generated results are visually pretty bad, so I'm wondering if there is something else missing.

Thanks!