when i put centerloss's parameters into optimizer, raise valueError("optimizer got an empty parameter list")
optimizer_centloss = torch.optim.SGD(criterion_cent.parameters(), lr=args.lr_cent)

请问，可以单独使用center loss不加softmax loss吗，效果怎么样

Gradient exploding when I set ‘weight-cent’ to 200+，then I got a loss value(Nan). why?

In my project experiment, the center loss does not decrease with the number of iterations.It seemingly irregular changes. I don't quite understand what's going on. I'd like to ask you about it.Thanks.

Packages:
torch 1.5.0
torchvision 0.6.0

Code:
distmat.addmm_(1, -2, x, self.centers.t())

Warning:
..\torch\csrc\utils\python_arg_parser.cpp:756: UserWarning: This overload of addmm_ is deprecated:
     addmm_(Number beta, Number alpha, Tensor mat1, Tensor mat2)
Consider using one of the following signatures instead:
     addmm_(Tensor mat1, Tensor mat2, *, Number beta, Number alpha)

Fix:
distmat.addmm_(x, self.centers.t(), beta=1, alpha=-2)

What are the axis in the plots shown in the README file?
How can we achieve it?

in the center_loss.py, the shape of ground truth label should be (batch_size), not num_classes
and I think it can be good, if you decrease using of cuda() in center_loss.py

Hi~
May I ask whether the implementation in this repo will be equivalent to the original implementation in paper?

# by doing so, weight_cent would not impact on the learning of centers
for param in criterion_cent.parameters():
     param.grad.data *= (1. / args.weight_cent)

Simply let alpha=alpha/lambda, (alpha is lr_cent, lambda is weight_cent), will it equivalent to previous implementation?

It seems the author do not adopt gradient w.r.t. c_j, and instead, use the delta rule shown below.

Hello authors and contributors of center loss, thanks for the impressive work. I got a question as I noticed the center update is based on each mini-batch, and if I had a small batch and many classes (way more than batch size), the center update may become tricky, and I wonder if using momentum or other optimizers is necessary for training. Thanks.

The center loss is not a regular change.

I don't know how Xi changes with depth.
请问，公式里说Xi是随深度变化的特征维度，代码里是怎么体现的，没看出来？feat_dim的设置是输入的维度吗，可以保证优化的是最后的输出维度吗？我用于的数据不是图片数据集。请大佬回答我的疑惑，谢谢。

Hello!
I am trying to train a model with central loss. My dataset has 901 classes. I am creating mini batches after shuffling the training data with batch size of 128 such that:

feats_dim = [128,512] per batch
labels_dim =[128] per batch

Then:
center_loss = CenterLoss(num_classes = ????, feat_dim = 512, use_gpu=False)

What should I pass in num_classes?
901: actual number of classes
128: classes in current batch

Hi, I tried to use your package directly in my network. However, the param.grad is NoneType and param.grad.data cannot be got.. Could you help me to explain it? Thanks!

I'm afraid that u're writing the wrong code for the classification loss. In pytorch, the softmax cross entropy loss should be

NLL + LogSoftmax

or

nn.CrossEntropyLoss + logits

Hello,thanks for your projects,when i use my own database something is wrong,the problems is:
Traceback (most recent call last):
File "main.py", line 213, in
main()
File "main.py", line 96, in main
train(model, criterion_xent, criterion_cent,optimizer_model, optimizer_centloss, train_loader, use_gpu, 2, epoch)
File "main.py", line 120, in train
for batch_idx, (data, labels) in enumerate(trainloader):
ValueError: too many values to unpack (expected 2)

I don't know how to fix it.thank you very much.

Congratuations! You have done a great work. But I have a question about the center loss.
Can you explain why you wrote there two lines?

Hi @KaiyangZhou,

Could you please kindly provide the code used for the visualization of the feature learning process?

请问 优化算法可以不用SGD用Adam吗

Hi, thanks for your source code. But I notice in the line:
https://github.com/KaiyangZhou/pytorch-center-loss/blob/master/center_loss.py#L47
you wrote the center loss as the mean of dis, however, in the paper, the center loss is just the sum of the dis, so could you explain the reason? Thanks a lot

A concise and easy to understand version

class CenterLoss(nn.Module):
    def __init__(self, num_class=10, num_feature=2):
        super(CenterLoss, self).__init__()
        self.num_class = num_class
        self.num_feature = num_feature
        self.centers = nn.Parameter(torch.randn(self.num_class, self.num_feature))

    def forward(self, x, labels):
        center = self.centers[labels]
        dist = (x-center).pow(2).sum(dim=-1)
        loss = torch.clamp(dist, min=1e-12, max=1e+12).mean(dim=-1)

        return loss

I use your center loss in my own vgg19. Although my loss decreasing slowly, but the acc on test set never changed, I'm sure I update both opimizer's params, what's wrong with my code?

Hi, Kaiyang. The feature embedding is 2-dim in this repository and it is straightforward to visualize.
But if the feature is in a high-dimension space (e.g 2048, ResNet50 GAP feature). How I visualize
it in 2-d feature space, should I utilize the approaches such as PCA or TSNE ?

Hi, would it be possible for you to share your pre-learned model? Thank you.

I want to use ResNet50 as my model and change Fully Connection layer to classify images.

#Create model:
model = models.resnet50(pretrained=True)
fc_inputs = model.fc.in_features
class FClayer(nn.Module):
def init(self):
super(FClayer, self).init()
self.fc1 = nn.Linear(fc_inputs, 2)
self.relu1 = nn.ReLU()
self.fc2 = nn.Linear(2, num_classes)
def forward(self, x):
x = self.relu1(self.fc1(x))
y = self.fc2(x)
return x, y
model.fc = FClayer()
...
#In train function:
features, outputs = model(inputs)
alpha = 1
loss = loss_criterion(outputs, labels) + (loss_criterion_cent(features, labels) * alpha)
optimizer.zero_grad()
optimizer_cent.zero_grad()
loss.backward()
optimizer.step()
for param in loss_criterion_cent.parameters():
param.grad.data *= (1./alpha)
optimizer_cent.step()

I dont know why when i was training, at the very first epochs, the accuracy had increased, but soon later, nothing happended. Here is my accuracy plot curve chart:

Do I have something wrong?

At the beginning of training, the center loss is very large, for example, 520, although the use of pre-trained model.
PS. I use the arcface as pre-train before adding center loss

Thanks the auther about the implementaion of center loss.But i have a question about will the feat_dim in center_loss have an negative effect on the result ? My experiment was on the res_net18 and the feat_dim is 2048,after using the center_loss,the accuracy has a sharp decrease from 80% to 60%.