An Object Detection Knowledge Distillation framework powered by pytorch, now having SSD and yolov5.

License: GNU General Public License v3.0

Python 100.00%

pytorch knowledge-distillation ssd object-detection deep-learning yolov5

object-detection-knowledge-distillation's People

Contributors

Stargazers

Watchers

object-detection-knowledge-distillation's Issues

使用retinaface-resnet50对rentinaface-mobile0.25进行蒸馏训练时，训练几个批次后loss_reg先变为Inf，其他损失函数变为NAN，想请问一下，一般是什么原因？不知道应该从哪方面改进

with different dataset

can I use different dataset instead of VOC，please ？

Yolov5 : teacher bounded regression loss, Lb

Thanks for the awesome implementation.
I had a look at the losses and for the yolov5 implementation:

this implementation does not have the teacher bounded regression loss.
Also, the classification loss term does not include the term for soft targets. As mentioned in the paper, "It is known that
a deep teacher can better fit to the training data and perform better in test scenarios. The soft labels
contain information about the relationship between different classes as discovered by teacher. By
learning from soft labels, the student network inherits such hidden information."

I wanted to ask if I am correct in pointing these out and if yes did you deliberately skipped these things?

Thanks

Why is the hint loss not implemented in the code?

how the dataset mean is computed?

Hello,
Can you please explain how these values were computed?

MEANS = (127, 127, 127)

I tried to compute them by myself on the VOC2007+2012 dataset, and I obtained these values:
(115.78, 110.57, 102.41)

thank you

hint loss is not work

the hint loss is too small to work and this loss is close to zero after few epochs quickly

您好，请问下fine-tune的时候，epochs最好设置为多少呢？还有我用vgg对mobilenet进行蒸馏训练的时候，loss基本降不去，请问有什么方式可以改进的呢？

hint loss 不是计算logits的L2距离吧

https://github.com/SsisyphusTao/SSD-Knowledge-Distillation/blob/0597fbee635afcf0b8710ba3a9e40ab9f010aea5/nets/multibox_loss.py#L158

torch.nn.modules.module.ModuleAttributeError: 'Detect' object has no attribute 'm'

No module named 'dcn_op_v2'

Dear @SsisyphusTao , thanks for starting this distillation approach with yolo v5.

I'm trying to run this on my computer, but I'm got this error: ModuleNotFoundError: No module named 'dcn_op_v2'.

I search on your others repositories, and I found the "Plugins for Pytorch & TensorRT". I've tried to install that, but I got another error: RuntimeError: Error compiling objects for extension.

Do you know how can I fix that?

Best regards,

Incorrect KLDivergen

https://github.com/SsisyphusTao/SSD-Knowledge-Distillation/blob/0597fbee635afcf0b8710ba3a9e40ab9f010aea5/nets/multibox_loss.py#L8

Bounded regression loss

https://github.com/SsisyphusTao/SSD-Knowledge-Distillation/blob/0597fbee635afcf0b8710ba3a9e40ab9f010aea5/nets/multibox_loss.py#L17-L23
F.mse_loss has reduction='mean' by default, but we should use reduction='sum' as for loss_l
https://github.com/SsisyphusTao/SSD-Knowledge-Distillation/blob/0597fbee635afcf0b8710ba3a9e40ab9f010aea5/nets/multibox_loss.py#L120
I think the next code corresponds to the formula from the article.

def bounded_regression_loss(Rs, Rt, gt, m, v=0.5):
    loss = torch.sum(F.mse_loss(Rs, gt, reduction='none'), 1)
    return torch.sum(loss * (loss + m > torch.sum(F.mse_loss(Rt, gt, reduction='none'), 1))) * v

SSD mbv2

Is there any reason why the mbv2 model take so much time to evaluate compared to the vgg model??

The given teacher checkpoint (vgg) doesn't fit with the current version of VGG in the repo.

size mismatch for loc.0.weight: copying a param with shape torch.Size([16, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([16, 1024, 3, 3]).
        size mismatch for loc.1.weight: copying a param with shape torch.Size([24, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([24, 512, 3, 3]).
        size mismatch for loc.2.weight: copying a param with shape torch.Size([24, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([24, 256, 3, 3]).
        size mismatch for loc.4.weight: copying a param with shape torch.Size([16, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([24, 256, 3, 3]).
        size mismatch for loc.4.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([24]).
        size mismatch for loc.5.weight: copying a param with shape torch.Size([16, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([24, 256, 3, 3]).
        size mismatch for loc.5.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([24]).
        size mismatch for conf.0.weight: copying a param with shape torch.Size([84, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([84, 1024, 3, 3]).
        size mismatch for conf.1.weight: copying a param with shape torch.Size([126, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([126, 512, 3, 3]).
        size mismatch for conf.2.weight: copying a param with shape torch.Size([126, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([126, 256, 3, 3]).
        size mismatch for conf.4.weight: copying a param with shape torch.Size([84, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([126, 256, 3, 3]).
        size mismatch for conf.4.bias: copying a param with shape torch.Size([84]) from checkpoint, the shape in current model is torch.Size([126]).
        size mismatch for conf.5.weight: copying a param with shape torch.Size([84, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([126, 256, 3, 3]).
        size mismatch for conf.5.bias: copying a param with shape torch.Size([84]) from checkpoint, the shape in current model is torch.Size([126]).

How to run with no ground-truth data

Hello,
Firstly, thank you very much for this great repository.
I am really interested to run your code on a video, where the teacher transfers his knowledge to the student. However, since I am running on a video, there are no ground truth data available like in VOC/COCO datasets. What would change in the loss if I only use information from the outputs of the student and the teacher? and how do you think this would affect the training?

Thank you very much for your help

自己实现该方法使用res50模型蒸馏的时候报显存不足

你好，我在retinanet上复现这个方法的时候，出现显存不足的问题，想请问下对于节约显存代码上有什么技巧么

RuntimeError: CUDA error: out of memory

Hi,

I always got the run time error after I ran python eval.py mbv2 --trained_model=checkpoints/student_mbv2_500_3934.pth. Does anyone know how to solve this error? Thanks.

ssisyphustao / object-detection-knowledge-distillation Goto Github PK

object-detection-knowledge-distillation's People

Contributors

Stargazers

Watchers

Forkers

object-detection-knowledge-distillation's Issues

Recommend Projects

Recommend Topics

Recommend Org