Class Counter in Sigmoid Cross Entropy Loss about hed HOT 9 CLOSED

I was not clear. By "iteration", I mean the iteration of the outer for-loop in the code I posted.
Since the counters were not reset between these iterations, each image in a batch ends up using a different beta, which is neither the neg/pos ratio within a batch, nor a neg/pos ratio within an image.

from hed.

The accumulation iters through all spatial locations in one SINGLE image. The counter resets everytime the forward function is called. The balancing factor we used is not precomputed across the whole dataset (for which it could be), but on a single image instead.

from hed.

Oooops. Thanks for the heads-up. You are correct. This is a bug. Luckily it doesn't affect the results: because it is in the forward loop only for "showing" the loss. And the (wrongly) computed balancing factor, should still be around 0.05vs0.95, similar to what is computed globally, or on a single image.

The crucial backward computation is correct.

Thanks again for noticing this... I'll correct it right away.

from hed.

quick question on the backward computation: on line 107:
bottom_diff[i * dim + j] *= 1 * count_neg / (count_pos + count_neg);

and line 110
bottom_diff[i * dim + j] *= count_pos / (count_pos + count_neg);

why there is an extra "1" in line 107? It looked like doesn't make any differences between these two lines while only bottom_diff is weighted based on the count of labels?

from hed.

Yes. You can remove the "1" and it's obviously equivalent.

from hed.

@ppwwyyxx
@lexie0212

Hi, the modified SigmoidCrossEntropy Loss layer has a line
that reads like -:

bottom_diff[i * dim + j] *= 1 * count_neg / (count_pos + count_neg);

It seems that the gradients are multiplied every iteration in the loops. It is a little confusing to me. Why there is a '*' ahead of the '=' ?

from hed.

That's back propagation.

from hed.

@ppwwyyxx
The loss may be something like:

So you mean the gradients calculation is automatically done by caffe, here in the code , the author is just multiplying the gradients by the weight factor?

from hed.

It's done in the code. https://github.com/s9xie/hed/blob/master/src/caffe/layers/sigmoid_cross_entropy_loss_layer.cpp#L89

from hed.