About unconditional training script about dit HOT 6 CLOSED

Hi @Suimingzhe. There's a few ways you could go about this. Possibly the easiest way would be to call train.py with --num-classes 1 and structure your dataset so that ImageFolder thinks there's only one class. You might also want to consider passing class_dropout_prob=0 when calling the DiT constructor. We didn't experiment much with unconditional models, so this is all somewhat experimental, but hopefully it's a good starting point.

from dit.

Thank you very much for such a quick reply! I will try your suggestion and thanks again.

from dit.

@wpeebles I notice you used the pre-trained autoencoder. If I want to train an uncondition DiT on FFHQ dataset, is it right that I need to train a new autoencoder first according to latent diffusion? Thanks.

from dit.

The VAE is unconditional, so you don't necessarily need to train a new one (although you could). We just reused the LDM/Stable Diffusion VAE models in our paper. It's a good idea to check that the pre-trained VAE gives good reconstructions on FFHQ before using it though.

from dit.

hi @Suimingzhe ， is it possible to get some feedback about training unconditional DiT? Are you successful? If yes, which way you use? Thanks

from dit.

hi @Suimingzhe ， is it possible to get some feedback about training unconditional DiT? Are you successful? If yes, which way you use? Thanks

Just as author suggested, you can：

1. set "num-classes=1", which means all your images belong to one class, so the shape of "embedding_table" is [1, hidden_size];
2. set "class_dropout_prob=0.0", which means you do not use classifier-free guidance for training, and you also should use "model.forward" rather than "model.forward_with_cfg" for sampling;

from dit.