Comments (12)
Hi,
I have tried DeepCluster on smaller datasets such as CIFAR10. Actually the method doesn't seem to work well in this setting.
I experimented with a Network In Network architecture.
With a careful cross validation of parameters (k=100) and whitening the images in input I obtained decent performance but way above the performance reported by RotNet (https://arxiv.org/abs/1803.07728).
For example, when training a SVM on top of block 2 of a NIN3 architecture I got 73% accuracy (RotNet: 82%; Random: 61%).
Hope it helps
from deepcluster.
@KANG-Xudong, which network are you using for the training? Is it your custom network or are you training CIFAR-10 on existing net?
from deepcluster.
@KANG-Xudong, which network are you using for the training? Is it your custom network or are you training CIFAR-10 on existing net?
I simply applied the AlexNet model in the /model/alexnet.py
from deepcluster.
I guess that might be the problem. In case you are using Alexnet
or VGG
then it would not result in learning of good representations. One of the major reasons being the image size. CIFAR-10 has image of size 32x32
and the above mentioned networks take in 224x224
as input. So you might define a smaller network for CIFAR-10 and then go ahead with the learning.
from deepcluster.
I guess that might be the problem. In case you are using
Alexnet
orVGG
then it would not result in learning of good representations. One of the major reasons being the image size. CIFAR-10 has image of size32x32
and the above mentioned networks take in224x224
as input. So you might define a smaller network for CIFAR-10 and then go ahead with the learning.
Thank you for your answer.
I have tried to resize the input image to the shape 244 x 244 to fit the network. Moreover, I have already tried to use this method (using the /models/alexnet.py as model) in supervised training, and get 81% accuracy with CIFAR-10 dataset.
Thus, I think that may not result from the image size.
from deepcluster.
You said you were using k=10
, but it need not be the case that the number of classes in your dataset would directly correspond to the number of clusters. You might need to try some ablations with the value of k
.
Also when you use k=10
, how many points are going in each cluster? Can you type in the stats here?
from deepcluster.
@mathildecaron31
Hi, I am working on a similar problem. Could you please provide more training information. Thanks!!
from deepcluster.
Hi,
I have tried DeepCluster on smaller datasets such as CIFAR10. Actually the method doesn't seem to work well in this setting.
I experimented with a Network In Network architecture.
With a careful cross validation of parameters (k=100) and whitening the images in input I obtained decent performance but way above the performance reported by RotNet (https://arxiv.org/abs/1803.07728).
For example, when training a SVM on top of block 2 of a NIN3 architecture I got 73% accuracy (RotNet: 82%; Random: 61%).
Hope it helps
Hi, Can you explain why this happens on small data sets? I also used the NIN architecture on CIFAR-10, but only got 7% higher classification results than random networks.
from deepcluster.
So, is there a conclusion about the effect on CIFAR10 dataset? I tried this method with several models like alexnet and resnet, but none of them show good results.
from deepcluster.
I have achieved 46.18 on cifar10, resnet18, 400 epochs, 100 clusters (the first conv is 3x3 and modified moco code, knn monitor), updated kmeans every 20 epochs. I do not think it is ok compared to moco ~90. Besides, the accuracy tends to gradually decrease...
from deepcluster.
I just want to add some extra experimental results to this thread. I've experimented on CIFAR-10 with VGG-11 and LeNet. I experimented with 50/100/100 clusters with pca-downampled to 32/64/128, and various training strategies. This method doesn't seem to adapt well. In the best setting (fine-tuning without fixing conv base), there are 2 percent drop in performance. I like the idea of DeepCluster very much but I really cannot figure out why DeepCluster wouldn't work in my case.
from deepcluster.
The approach of deep cluster does not adapt to Cifar-10 dataset. I tried by resizing the images to match the alexnet/vgg16 input size, but model does not learn anything. The debugging showed that there are some nan/zero predictions from the model during the differerent batches of input data. This halts the loss and does not decrease.
Conclusion: Deepcluster does not adapt to Cifar10.
from deepcluster.
Related Issues (20)
- AttributeError: 'Clustering' object has no attribute 'obj' HOT 3
- RuntimeError: invalid argument 5: k not in range for dimension at /pytorch/aten/src/THC/generic/THCTensorTopK.cu:23 HOT 2
- train dataset that is not sorted in different folders HOT 1
- Hey, can you please share how you have solved this problem because I am also getting the same type of error which m not able to solve.hope you can help me out. Thank you. HOT 2
- Do I need labels or pseudo-labels as input for clustering? HOT 8
- A problem about the clusters and pseudolabels every epoch HOT 8
- A problem why use different optimizer for top layer and other layers HOT 1
- About the clustering loss HOT 1
- About the clustering loss HOT 3
- Is this model universal on different datasets? HOT 1
- Why does the deep cluster model cluster unevenly when using MNIST datasets? HOT 1
- Dealing with empty clusters HOT 3
- Gradcam
- about NMI
- Could I use it for image color clustering?
- PackagesNotFoundError
- Faiss
- how to avoid empty clusters. HOT 1
- Loading
- The pre-trained AlexNet on ImageNet
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepcluster.