Exercise solutions for Chapter 11 - Batch Normalization fix about handson-ml HOT 5 CLOSED

Yikes! You are absolutely right, good catch! I'll update the solution, thank you so much.

And also thanks for your very kind words, I'm very touched. It's fantastic to have such enthusiastic and helpful readers! :))

Cheers,
Aurélien

from handson-ml.

Just fixed this. Indeed, Batch Normalization manages to reach 99.4% accuracy, instead of 99.32%. Not a huge gain, but still nice to have. Thanks again!

from handson-ml.

I found that such a margin rather depends on the initial state of the randomizer, so yes, Batch Normalization doesn't really give any advantage in this case, but it doesn't spoil anything either.

BTW, in the Tensorflow's documentation, they mentioned that you can add a dependency to the training operation so that it also triggers extra_update_ops:

optimizer = self.optimizer_class(learning_rate=self.learning_rate)
update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
    training_op = optimizer.minimize(loss)

In this case, you don't even need to retrieve extra_update_ops from the graph and explicitly call sess.run(extra_update_ops, feed_dict=feed_dict)

I found it here: https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization

It might be something they added recently

Thank you! It's my pleasure! :)

from handson-ml.

Indeed, this does simplify the code. I mention this in the notebook for chapter 11, just before cell [36], but perhaps I should also mention it more prominently in the book (currently, the control_dependencies() function is just mentioned briefly in chapter 12).
Thanks again for your feedback.

from handson-ml.

Oh, sorry! It was my bad - I did not notice that 😰

from handson-ml.