Comments (2)
With the batch all strategy, since we only take the average loss over the semi-hard and hard triplets it's totally normal that the loss doesn't decrease.
However if the loss gets stuck at exactly the margin (0.5
), it indicates that all the embeddings are collapsed into a single point. One solution is to reduce the learning rate until training does not collapse.
In your case I'm not sure what happens, maybe try to decrease the learning rate even further and see if you can get out of this collapsing behavior (with the batch all strategy)?
Also my code currently only works because MNIST has only 10 classes. The model currently receives batches with random images. As I have 64 images in a batch and 10 classes, I have a lot of triplets to work with. However in your case you have 3000 classes so there is only a small probability that you will get informative triplets in a batch of size 64.
The solution is to create your own batches (for instance 16 different classes with 4 images each for a total batch size of 64) and feed them to the model.
Edit: adding issue to track progress on this in #7
from tensorflow-triplet-loss.
Thanks for your reply.
from tensorflow-triplet-loss.
Related Issues (20)
- What does 'embedding_mean_norm' mean? HOT 2
- loss=0 in step=101(after two step) HOT 1
- Saving weights of model and calculation of embeddings HOT 1
- Using batch_all_triplet_loss function HOT 2
- Embeddings Collapse very fast
- fraction_positive increasing HOT 4
- the function of _get_triplet_mask in triplet_loss.py
- base_model
- Multi domain triplet loss
- Why average loss value by `batch_size` when using `batch_hard` method?
- PyTorch Implmentation of triplet loss
- the difference between tf.contrib.losses.metric_learning.triplet_semihard_loss and batch_all_triplet_loss
- Implementation of metrics to monitor training process in tf.keras environment
- Advice on which loss to optimize HOT 2
- Performance issues in the program
- Performance issue in /model/tests (by P3) HOT 1
- Got Nan value when label's id is greater than 159 HOT 1
- hard triplet convergence!
- Some corrections in MNIST_dataset since TF 2.0
- OSS License compatibility question
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorflow-triplet-loss.