Meaning behind --update-mean-var --train-beta-gamma about icnet-tensorflow HOT 3 CLOSED

Hey @Tamme, let me explain the batch normalization layer first. There are four variables in the batch normalization layer, and they are moving_mean, moving_variance, gamma, and beta respectively. And moving_mean and moving variance are not trainable variables, so we need to update them using update ops which is put in tf.GraphKeys.UPDATE_OPS, you can take a look at tensorflow docs. So I use the flag --update-mean-var to decide whether to update mean and var ( Since update them in large batch size is better, if we train in mini-batch, we can frozen these two variables for better reuslts).

from icnet-tensorflow.

hey, when I run the training without -update-mean-var the evaluation results are almost zero. Do you know why @hellochick ?

from icnet-tensorflow.

@manuel-88, if you didn't update mean and variance, then the batch normalization layer will do nothing. Maybe this is the problem.

from icnet-tensorflow.