L2 normalization about tensorflow-triplet-loss HOT 2 CLOSED

It is possible to add normalization at this point in the model_fn function:

# -----------------------------------------------------------
# MODEL: define the layers of the model
with tf.variable_scope('model'):
    # Compute the embeddings with the model
    embeddings = build_model(is_training, images, params)

    # Optionally L2 normalize the embeddings
    if params.l2_normalize:
        embeddings = tf.nn.l2_normalize(embeddings, axis=1)

I followed the paper In Defense of the Triplet Loss for Person Re-Identification when I did it without L2 normalization:

We did not use a normalizing layer in any of our final experiments. For one, it does not dramatically regularize the network by reducing the available embedding space: the space spanned by all D-dimensional vector of fixed norm is still a D − 1- dimensional volume. Worse, an output-normalization layer can actually hide problems in the training, such as slowly collapsing or exploding embeddings.

Concerning the margin, a default margin of 0.5 usually works well even if the embeddings are not constrained to the L2 unit sphere.

from tensorflow-triplet-loss.

Thanks for the code and the explanation. That's interesting.

I haven't read the paper, yet, but will do!

from tensorflow-triplet-loss.