View Code? Open in Web Editor
NEW
This project forked from zssloth /embedded-neural-network
collection of works and self-notes of neural network based embedded applications
Ruby 16.71%
HTML 24.27%
CSS 59.02%
neural-network-notes's Introduction
This is a collection of papers aiming at reducing model sizes or the ASIC/FPGA accelerator for Machine Learning, expecially deep neural network related applications. (Inspiled by Neural-Networks-on-Silicon )
Notes can be found in my personal blog . (TODO )
structured matrices
Structured Convolution Matrices for Energy-efficient Deep learning. (IBM Research–Almaden)
Structured Transforms for Small-Footprint Deep Learning. (Google Inc)
An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections.
Hashing
Functional Hashing for Compressing Neural Networks. (Baidu Inc)
Compressing Neural Networks with the Hashing Trick. (Washington University + NVIDIA)
Learning compact recurrent neural networks. (University of Southern California + Google)
Teacher-Student Mechanism (Distilling)
Distilling the Knowledge in a Neural Network. (Google Inc)
Fixed-precision training and storage
Deep neural networks are robust to weight binarization and other non-linear distortions. (IBM Research–Almaden)
XNOR-Net, Ternary Weight Networks (TWNs), Binary-net and their variants.
Recurrent Neural Networks With Limited Numerical Precision. (ETH Zurich + Montréal@Yoshua Bengio)
Neural Networks with Few Multiplications. (Montréal@Yoshua Bengio)
1-Bit Stochastic Gradient Descent and its Application to Data-Parallel Distributed Training of Speech DNNs. (Tsinghua University + Microsoft)
Sparsity regularizers & Pruning
Learning both Weights and Connections for Efficient Neural Networks. (SongHan, Stanford University)
Deep Compression, EIE. (SongHan, Stanford University)
Dynamic Network Surgery for Efficient DNNs. (Intel)
Compression of Neural Machine Translation Models via Pruning. (Stanford University)
Accelerating Deep Convolutional Networks using low-precision and sparsity. (Intel)
Low-rank matrix factorization
Learning compact recurrent neural networks. (University of Southern California + Google)
Adaptive Computation Time for Recurrent Neural Networks. (Google DeepMind@Alex Graves)
Benchmark and Platform Analysis
Fathom: Reference Workloads for Modern Deep Learning Methods. (Harvard University)
DeepBench: Open-Source Tool for benchmarking DL operations. (svail.github.io-Baidu)
Recurrent Neural Networks
FPGA-based Low-power Speech Recognition with Recurrent Neural Networks. (Seoul National University)
Accelerating Recurrent Neural Networks in Analytics Servers: Comparison of FPGA, CPU, GPU, and ASIC. (Intel)
Accelerating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and ASIC. (Intel)
Convolutional Neural Networks
Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent