Comments (4)
This implementation utilizes the straight-through gradient estimator, which is also utilized by GDAS.
from dada.
@latstars . Thank you for your answer. I understand. I have another question.
I tried to search the policies using larger ImageNet data where I used 500 classes and 30K+ images. The policies are so weird. But I observed that when I use a lower number of classes and images I get policies like what you have in your genotype.py.
Maybe, it is not the right way to search the policy. But I want to know the reasons why my policies look like this'. If you have any ideas about what causes this kind of policies, pls share them with me,
from dada.
@latstars . Thank you for your answer. I understand. I have another question.
I tried to search the policies using larger ImageNet data where I used 500 classes and 30K+ images. The policies are so weird. But I observed that when I use a lower number of classes and images I get policies like what you have in your genotype.py.
Maybe, it is not the right way to search the policy. But I want to know the reasons why my policies look like this'. If you have any ideas about what causes this kind of policies, pls share them with me,
Hi, Raja. Maybe you use a larger dataset, then the iteration is more than the small dataset. However, more iteration will lead to over-optimize, since there is no l2-normalization (weight-decay). I suggest that you can try to search the policy with less epoch or smaller learning rate for augmentation parameters.
from dada.
@latstars . Hi. Thank you for your answer. This is impressive. When I reduced the learning rate the policies seem good. In order to validate it, I have conducted several experiments by varying learning rates from 0.5 to 0.0001 where 0.5,0.2,0.1,0.002,0.005 are not working well. But, 0.001, 0.0001,0.0002 are fine (Working fine means that the policies look like your experiment policies --> Just a visual inspection). Now, I'm facing another question. Which policy is good.? How can I determine the policies are good.? Any ideas you can give?
The following result is an example of experiments with a learning rate of 0.0001.
from dada.
Related Issues (20)
- About method DifferentiableAugment HOT 5
- Update install instruction b to use == to specify the version of cudatoolkit instead of =
- How to add (weight, probability and magnitude) to the forward calculation to calculate the gradient? HOT 3
- No genotypes module HOT 1
- How to choose the final policy in the search phase? HOT 4
- Why in search-gumbel-architect.py-def _backward_step_unrolled(), dalpha is []? HOT 1
- Why you put two "elif dataset == 'reduced'": HOT 2
- why did you produce 5 splits in "sss = StratifiedShuffleSplit(n_splits=5, test_size=split, random_state=0)"
- A question about the gradients of sampling HOT 1
- Low accuracy while searching HOT 6
- use one-step unrolled validation loss HOT 3
- Ops weights and probabilities not getting updated HOT 5
- Reduced ImageNet split HOT 2
- Error in search_gumbel HOT 2
- How to get the NetworkCIFAR ? HOT 2
- Explaination for Equation 10
- dataloader num_workers=0 HOT 6
- Could you provide ILSVRC/ImageSets/CLS-LOC/train_cls.txt? HOT 1
- Extract the found policy HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dada.