rgb info in the generator is not collected in the correct way? about gigagan-pytorch HOT 8 CLOSED

Great! Thanks! BTW, I see that you upload some codes for upsampler, however it looks like it is a simple Unet architecture with ResNet blocks. Do you plan to build the unet_upsampler with gigaGAN blocks recently?

from gigagan-pytorch.

Thanks! Yeah I agree the info in the paper is insufficient especially for the upsampler part. But my two cents on the potential architecture:

1. Since in table A.2, the configuration of upsampler model has mapping network, w dimension, etc, I think definitely there is latent variable in the upsampler, similar to the base model. So the mapping network with concatenation of noise and global text embedding as input, as well as the adaptive filter with style modulation should be there. However, in the main text, it mentioned "residual blocks", so I guess it is more like a resnet block, but with conv layers replaced by adaptive kernel conv.
2. Since for text-to-image upsampler, the multi-scale loss is also True, there should be multi-scale outputs (i.e., toRGB layers). It makes no sense to have toRGB layers on resolutions lower than the input, so I guess you can have toRGB layers on the last few Unet blocks where the resolution is larger or equal to the input resolution.

from gigagan-pytorch.

@XavierXiao oh yes 🤦‍♂️ thank you!

from gigagan-pytorch.

@XavierXiao i'm kind of confused by what they used for upsampling

they have no architectural diagram, and only said it was a traditional unet, so that's what i'm going to start off with, paired with the GigaGAN discriminator. i was going to modify the unet to also output the rgb, like the style/gigagan generator

from gigagan-pytorch.

@XavierXiao open to pointers and suggestions, if you have any insights

from gigagan-pytorch.

@XavierXiao yes what you say makes sense Xavier! will build it exactly how you said!

from gigagan-pytorch.

@XavierXiao b22ecfc let me know if we are on the same page after this commit

from gigagan-pytorch.

Great! Looks good to me! Only inconsistency to paper that I noticed is that the paper mentioned (in 3.4) the Unet is asymmetric (i.e., 3 down blocks and 6 up blocks for a 8x up-sampler), and the skip connection of the unet is, of course, only at matched resolutions of up and down blocks. Here your Unet seems to be symmetric. Not sure if it matters though.

from gigagan-pytorch.