Comments (4)
Yes, you can specify different learning rate for each parameter, by scaling its gradient. You can overwrite the get_gradient_processor()
method in your Model with something like:
def get_gradient_processor(self):
return [ScaleGradient([('conv.*/W', 0.5), ('fc.*', 2)])]
ScaleGradient takes a list of (regex, multiplier)
tuple. Parameters that are scaled will get printed out so you could check.
// UPDATE: this usage was later deprecated. See comment below.
from tensorpack.
Exactly what I needed. Thank you!
from tensorpack.
I couldn't find any function as get_gradient_processor()
in the base class ModelDesc
. I tried adding this function into my Model, but it seemed that this function was never called.
from tensorpack.
This usage was deprecated one year ago. Now gradient processors can be added onto optimizer with http://tensorpack.readthedocs.io/en/latest/modules/tfutils.html#tensorpack.tfutils.optimizer.apply_grad_processors .
from tensorpack.
Related Issues (20)
- Why pycocotools has no author, licence info neither a github repo or official website? HOT 2
- Performance issues in the program HOT 1
- Activation function for last layer in DoReFaNet HOT 2
- Resnet DoReFaNet implementation HOT 1
- Applying quantization to the CNN for CIFAR net with BNReLU gives 0.1 validation accuracy HOT 1
- Exporting detectron2 model to pkl file HOT 3
- Tensorflow summaries HOT 1
- using tensorpack.dataflow and not gaining any output
- Saving lmdb file using LMDBSerializer.save leads to lmdb with one entry HOT 1
- No access to models.tensorpack.com HOT 3
- AttributeError: 'MultiThreadMapData' object has no attribute '_evt'
- InvalidArgumentError: Default MaxPoolingOp only supports NHWC on device type CPU. Shufflenet-V1 eval CPU only. HOT 1
- Dataset 'train' is already registered!
- Site not accessible - http://models.tensorpack.com/ HOT 6
- Option to disable the tqdm progress bars
- Please read & provide the following information
- Please read & provide the following information
- Run code on CPU
- Can i use it in win 10?
- nr_tower
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorpack.