welch-lab / michigan Goto Github PK

Hi i'm Dat Nguyen CS student who currently research on VAE and detect underlying pattern of single cell data. I did review your code and notice you have similar way of calculate loss of VAE.

Pytorch code

reproduction_loss = nn.functional.mse_loss(x_hat, x, reduction='none').sum(dim=1).mean()
KLD = torch.mean(-0.5 * torch.sum(1 + log_var - mean **
                     2 - log_var.exp(), dim=1), dim=0)

But the problem for this is reproduction_loss is too big since we only reduce it to mean ( batch size )

So my question is how could you interpret those loss and make sure that the model going to correct direction.

Hi! Thank you for sharing your codes!

When I look at your beta-TCVAE implementation, I found your TC loss is negative in FunctionProgramExamples/examples/example_beta_TCVAE.ipynb.

epoch: 98; iteration: 3168; beta-TCVAE loss:4330.94580078125; Total Correlation:-0.8152694702148438
epoch: 99; iteration: 3200; beta-TCVAE loss:4333.12744140625; Total Correlation:-1.0267829895019531

However, in principle, TC loss should be positive.
Why is your TC loss negative?

Best,