Departures from paper wrt discriminator loss, generator first layer and activation functions? about gain HOT 3 CLOSED

Hello,

Below is my answer for your questions.

1. *2 comes from the fact that in this example, we use 50% missing rate; thus, to compensate that, I put *2. However, note that it is not crucial and you can remove this.

2. As can be seen in Figure 1 in the paper, the inputs of the generators are [x,m,z].

• z comes with x using m*x + (1-m)*z.
• Then, m is concatenated.
• In the case of auto-encoder type of neural networks, you don't need to use the same size of input and output.

3. The activation function for this MNIST example is relu. For the other experiments, we use tanh.

Thanks.

from gain.

Thanks for your prompt response!

If I may, I still don't see the point of "*2" as, according to the paper, the discriminator is supposed to be trained at the datapoint(s) where b_j=0

Regarding the second point, I do know that generally speaking input and output are not restricted to share the same dimensions (which is also the case for GANs where the dimensionality of input noise is in principal substantially greater than that of the real data)
All I am saying is that based on the paper, I wouldn't suspect that M is needed in such a way (i.e apart from m*x + (1-m)*z)

Best,
Maria

from gain.

1. You can just ignore "*2". Not much meaningful.
2. Additional "m" is needed in order to distinguish which component comes from x and which component comes from z. Without "m", it is not possible to distinguish these two in terms of the generator.

from gain.