Cannot find L1 loss in training code about network-slimming HOT 4 CLOSED

https://github.com/Eric-mingjie/network-slimming/blob/master/main.py#L126

from network-slimming.

the loss we find in main.py is
def updateBN(): for m in model.modules(): if isinstance(m, nn.BatchNorm2d): m.weight.grad.data.add_(args.s*torch.sign(m.weight.data)) # L1
the function torch.sign() returns -1 or 1,why not use function abs() as what is said in the paper:

In our experiment, we choose g(s) = |s|, which is known as L1-norm and widely used to achieve sparsity.
@Eric-mingjie

from network-slimming.

The derivative of abs() function is the sign() function.

from network-slimming.

为什么不直接在loss定义正则化项, 而是直接写在梯度反传上呢? 这样做两者是等价还是在某些情况下有区别啊? 比如采用不同的optimizer?

from network-slimming.