yirongmao / softmax_variants Goto Github PK

In the following two lines from class LGMLoss(nn.Module),
log_covs = torch.unsqueeze(self.log_covs, dim=0)
covs = torch.exp(log_covs) # 1cd

the covariance matrix is considered as an identity matrix even though this is the implementation of the original paper whereas in the class LGMLoss_v0(nn.Module), (mentioned as 'LGMLoss whose covariance is fixed as Identity matrix' in the comments) no covariance matrix was considered.

在train_mnist_LGM_u.py中用 margin_logits过了一个 nn.CrossEntropyLoss

我根据您的代码推了一下公示，想确认一下是不是 margin_logits 过一层 LogSoftmax之后才等于原论文中的 eq.17 ？

I am not very familiar with the change of the loss function, how to solve this problem?
error：optimizer_l = optim.SGD(lgm_loss.parameters(), lr=0.1)

print('Epoch [%d], Iter [%d/%d] Loss: %.4f Acc %.4f'
% (epoch, i + 1, len(train_loader) // batch_size, loss.data[0], accuracy))

gave an error,
*** IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number
Replacing loss.data[0] to loss.item() has solved the issue.

what is feat_dim in LGMLOSS_v0,How can I set it

Im intersted in the paper: https://arxiv.org/abs/1901.07711

You have done a great work.

But I have a question about the LMCL_loss. How to understand the centers as show bellow:

I think thay are stand for the parameters of the last fully connected layer. But I wonder how to upodate them. In your code, I think they are fixed values, are they?

Looking forward to your reply.

Hi,
According to LMCL paper, the margin was only computed between feature x and its own class weight W.
Hence I think label should be passed into your implementation and LMCL_loss should be modified.

LGM loss训练时大概20epoch 以后loss为负值，一直持续到结束，但是测试时效果还不错，请问loss的负值如何解决呢，期待您的回复！