emited / variationalrecurrentneuralnetwork Goto Github PK

Or am I wrong?

Hi,

Can you attach a license for the code so it's easier for others to reuse? Thank you very much.

the code
def _nll_bernoulli(self, theta, x): return - torch.sum(x*torch.log(theta + EPS) + (1-x)*torch.log(1-theta-EPS)) may got nan loss.
i think it should be
def _nll_bernoulli(self, theta, x): return - torch.sum(x*torch.log(theta + EPS) + (1-x)*torch.log(1-theta+EPS))

Train Epoch: 3 [0/60000 (0%)] KLD Loss: 2.687659 NLL Loss: 73.599564
Train Epoch: 3 [2800/60000 (21%)] KLD Loss: 2.976363 NLL Loss: 78.757454
Train Epoch: 3 [5600/60000 (43%)] KLD Loss: 2.837864 NLL Loss: 78.958122
Train Epoch: 3 [8400/60000 (64%)] KLD Loss: nan NLL Loss: nan
Train Epoch: 3 [11200/60000 (85%)] KLD Loss: nan NLL Loss: nan
====> Epoch: 3 Average loss: nan
====> Test set loss: KLD Loss = nan, NLL Loss = nan
Train Epoch: 4 [0/60000 (0%)] KLD Loss: nan NLL Loss: nan

Hi,

in the VRNN class (in model.py), you cut the gradients behind h_{t-1} using the _repackage_state() function.
I've been thinking about this question for a while now and would have said that the correct thing to do is to not cut the gradients because nothing in the paper indicates that one should.

May I ask what your reasoning is? - I'm not very sure about mine.

Thanks!
Best, Max

When load the data from the dataloader, the dim is transposed:
data = Variable(data.squeeze().transpose(0, 1)).to(device)
So in model.py line 82

 for t in range(x.size(0)):

	phi_x_t = self.phi_x(x[t])

the size of phi_x_t is ( batch_size, h_dim ), is that correct ?

Error message :

Done!
Traceback (most recent call last):
File "train.py", line 112, in
train(epoch)
File "train.py", line 25, in train
data = (data - data.min().data[0]) / (data.max().data[0] - data.min().data[0])
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

so , I changed the code (as below)

data.min().data[0] -> data.min()

*.data[0] -> *.item() ( * Denotes all variables that use data [0].)

Does this deviate from your intention in your code? @emited