dongyp13 / stochastic-quantization Goto Github PK
View Code? Open in Web Editor NEWTraining Low-bits DNNs with Stochastic Quantization
Training Low-bits DNNs with Stochastic Quantization
Hi . Thank you for provide your great code!
I have a two question.!
In paper, vgg9 model(cifar10) don't use data augmentation. But , this code use data augmentation like crop and mirror. In paper vgg 9 get 10% error in cifar 10. Is this the result of using data augmentation? ...
Is this code saved as a 32bit model or binary model ?
Thank you !!
@dongyp13 Hi, I read your paper and thanks for sharing code.
I am now trying to reproduce your paper with tensorflow. Since I do not know anything about caffe, so I just need to reproduce it with the information only on paper.
My question is,
Again, thanks for sharing the code and congratulations on your acceptance at the BMVC conference.
Best Regards.
@dongyp13 Hi, thanks for sharing code. Could you release your pretrained models (specifically ImageNet AlexNet/ResNet SQ-TWN)? I try to reproduce your results but accuracy is very low even at first iterations...
Hi, 您好:
看了您的论文以及源码,获益颇多,在此有几个问题想请教一下,还请帮忙解惑~
1. 权重量化为低比特,input 以及features map是float32,是否有什么方法也可以做到量化?我想过直接映射到8bit,但是与-1这样的权重乘积会出错,除非把权重量化到8bit,然后扩成16bit计算,防止溢出,效率会下降,而且每层还需要取max和min。
2. bias,batchnorm,prelu这些一个参数作用于整个feature map,量化的话,是否会有很大损失?
Wow.很棒的工作!
我现在是仿照您的工作自己实现了下BWN,我浏览下了您的代码,貌似在梯度更新的时候并没有按照paper里推导公式来更新的 是直接将算出的梯度更新到原来的权重里的,或者代码是在其它的地方实现了这一更新吗
我最近在研究量化相关的方向,看到了您的论文。注意到您使用ResNet-56在cifar100上取得了35.01的错误率,但我自己实现时最高只有43。
我去看了您的resnet56的代码,但因为我没学过caffe所以看不懂。
想请教您一下,您选取的optimizer、学习率退火方式以及相应的超参数是什么?
还有,我看您在别的issue里提到您只量化了weight,请问这里的weight包括bn层及bias吗?
十分感谢
Hi @dongyp13,
Thank you for sharing the code.
I am training the ImageNet/AlexNet-BN/SQ-TWN from scratch. The network seems to be learning, 50K iterations of the first stage have run so far, and I can see the Top1/Top5 accuracy as 0.12/0.28.
I extracted the trained weights from this caffemodel snapshot, and was expecting that 50% (= r) of the weight values would be +1/0/-1, but I dont see such quantized weights for any conv/fc layer in the caffe model.
Could you please guide from where to obtain the quantized weights? Is there any special way to dump them, or will they get reflected in the caffe model after all the stages have run?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.