Hi, thanks for the great work.

I'm trying to reproduce the experiment on pointwise constraints.
However, it seems the training converges too slowly specifically on discriminator.json, in contrast to other configurations.
After 7 days of training on 2 V100 gpus, I found the Eval/b(x)_mean is still fluctuating around 0.25~0.28 (compared to the result reported on paper). It seems Eval/KL(p || pi) is still decreasing, so I believe it's not converged yet.

Are there any differences between the configuration on this repo and the one the model that gives the result on the paper is trained with?
Or, could you give any recommendation to boost the training?

Thanks in advance!

Hi ! I have a clarification question about batch_size and forward_batch_size in the experiment configurations.

The paper mentions that all experiments use batch size of 2048. This large batch size (together with a forward_batch_size of 256 as in configs/gdc/pointwise/word.json) requires full utilization of 48G of GPU memory and is unfortunately a bit too much for my computation device (I only have 12 G GPUs, sad ...).

That said, I also noticed that the batch_size and forward_batch_size is reduced to 64 and 32, respectively, in configs/gdc/pointwise/wordlist.json, while the rest of the configurations remain the same as in word.json. Using these smaller batch numbers, I was able to comfortably use my humble GPUs to train the GDC model.

My question is: how does the batch_size and forward_batch_size impact the results? Do you think I can reach similar performance if I use the provided wordlist.json configuration as is (i.e., without changing anything in the file), and in particular, with the smaller batch numbers (64 and 32, respectively)? Also, I am aware that I can use 2048 for batch_size and a small forward_batch_size such as 32, but this runs extremely slowly... 64 and 32 are preferred for my use case.

Thanks in advance! By the way, great work and thanks for sharing the code!