Topic: knowledge-distillation Goto Github

👇 Here are 449 public repositories matching this topic...

1duo / awesome-ai-infrastructures
knowledge-distillation,Infrastructures™ for Machine Learning Training/Inference in Production.
User: 1duo
artificial-intelligence machine-learning deep-learning machine-learning-systems awesome-list deep-learning-framework kubernetes apache-spark apache-arrow apache-mesos

aberhu / knowledge-distillation-zoo
knowledge-distillation,Pytorch implementation of various Knowledge Distillation (KD) methods.
User: aberhu
kd-methods knowledge-transfer knowledge-distillation kd teacher-student model-compression distillation

aimagelab / mammoth
knowledge-distillation,An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning
Organization: aimagelab
pytorch continual-learning neurips2020 knowledge-distillation deep-learning der experience-replay dark-experience-replay

alibaba / easynlp
knowledge-distillation,EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
Organization: alibaba
transformers bert nlp pretrained-models deep-learning pytorch fewshot-learning knowledge-distillation knowledge-pretraining text-image-retrieval

alibaba / easytransfer
knowledge-distillation,EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
Organization: alibaba
Home Page: https://www.yuque.com/easytransfer/cn/
bert nlp-applications knowledge-distillation transfer-learning

chaitjo / efficient-gnns
knowledge-distillation,Code and resources on scalable and efficient Graph Neural Networks
User: chaitjo
Home Page: https://www.chaitjo.com/post/efficient-gnns/
graph-neural-networks graph-representation-learning graph-convolutional-networks gnn efficient-deep-learning efficient-neural-networks scalable-deep-learning contrastive-learning knowledge-distillation

clovaai / overhaul-distillation
knowledge-distillation,Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
Organization: clovaai
knowledge-distillation knowledge-transfer teacher-student network-compression iccv2019

czg1225 / slimsam
knowledge-distillation,SlimSAM: 0.1% Data Makes Segment Anything Slim
User: czg1225
knowledge-distillation model-compression model-pruning segment-anything-model

dkozlov / awesome-knowledge-distillation
knowledge-distillation,Awesome Knowledge Distillation
User: dkozlov
knowledge-distillation knowledge-transfer teacher-student co-training distillation distillation-model model-distillation knowldge-distillation kd model-compression

haitongli / knowledge-distillation-pytorch
knowledge-distillation,A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
User: haitongli
pytorch knowledge-distillation deep-neural-networks cifar10 model-compression dark-knowledge computer-vision

hikaritju / ld
knowledge-distillation,Localization Distillation for Object Detection (CVPR 2022, TPAMI 2023)
User: hikaritju
detection knowledge-distillation ld object-detection pytorch deep-learning

horseee / awesome-efficient-llm
knowledge-distillation,A curated list for Efficient Large Language Models
User: horseee
compression knowledge-distillation language-model llm model-quantization pruning-algorithms efficient-llm llm-compression

hoytta0 / knowledgedistillation
knowledge-distillation,Knowledge distillation in text classification with pytorch. 知识蒸馏，中文文本分类，教师模型BERT、XLNET，学生模型biLSTM。
User: hoytta0
knowledge-distillation bert pytorch model-compression distillation

huawei-noah / efficient-computing
knowledge-distillation,Efficient computing methods developed by Huawei Noah's Ark Lab
Organization: huawei-noah
knowledge-distillation model-compression binary-neural-networks pruning quantization self-supervised

huawei-noah / pretrained-language-model
knowledge-distillation,Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
Organization: huawei-noah
knowledge-distillation model-compression quantization pretrained-models large-scale-distributed

idea-research / dwpose
knowledge-distillation,"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)
Organization: idea-research
controlnet knowledge-distillation pose-estimation stable-diffusion-webui

ilovepose / fast-human-pose-estimation.pytorch
knowledge-distillation,Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
Organization: ilovepose
human-pose-estimation deep-learning coco-keypoints-detection mpii-dataset mscoco-keypoint fast-pose-distillation knowledge-distillation

imirzadeh / teacher-assistant-knowledge-distillation
knowledge-distillation,Using Teacher Assistants to Improve Knowledge Distillation: https://arxiv.org/pdf/1902.03393.pdf
User: imirzadeh
deep-learning knowledge-distillation neural-network-compression

intel / neural-compressor
knowledge-distillation,SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
Organization: intel
Home Page: https://intel.github.io/neural-compressor/
low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models

jingyi0000 / vlm_survey
knowledge-distillation,Collection of AWESOME vision-language models for vision tasks
User: jingyi0000
computer-vision deep-learning knowledge-distillation survey transfer-learning vision-language-model clip multi-modal-model

karanchahal / distiller
knowledge-distillation,A large scale study of Knowledge Distillation.
User: karanchahal
knowledge-distillation feature-distillation pytorch-lightning

lenscloth / rkd
knowledge-distillation,Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
User: lenscloth
deep-neural-networks computer-vision metric-learning knowledge-distillation deep-learning

lhyfst / knowledge-distillation-papers
knowledge-distillation,knowledge distillation papers
User: lhyfst
knowledge-distillation model-compression paper dark-knowledge reading-list

lirongwu / kdga
knowledge-distillation,Code for NeurIPS 2022 paper "Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks"
User: lirongwu
graph-augmentation graph-neural-networks graph-structure-learning knowledge-distillation

ltkong218 / ifrnet
knowledge-distillation,IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation (CVPR 2022)
User: ltkong218
real-time slow-motion frame-interpolation deep-networks pytorch deep-learning video optical-flow knowledge-distillation view-synthesis

lxztju / pytorch_classification
knowledge-distillation,利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码
User: lxztju
pytorch image-classification deployment svm knn cnn label-smoothing densenet resnext resnet flask random-forest knowledge-distillation

megvii-research / mdistiller
knowledge-distillation,The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf
Organization: megvii-research
pytorch knowledge-distillation computer-vision deep-learning cifar coco cvpr2022 imagenet iccv2023

microsoft / cream
knowledge-distillation,This is a collection of our NAS and Vision Transformer work.
Organization: microsoft
nas automl vision-transformer rpe vit-compression efficiency knowledge-distillation

microsoft / neuronblocks
knowledge-distillation,NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
Organization: microsoft
question-answering deep-learning pytorch natural-language-processing text-classification artificial-intelligence dnn qna text-matching knowledge-distillation

mingsun-tse / efficient-deep-learning
knowledge-distillation,Collection of recent methods on (deep) neural network compression and acceleration.
User: mingsun-tse
model-compression network-pruning knowledge-distillation deep-learning deep-neural-networks efficient-deep-learning

nathanhubens / fasterai
knowledge-distillation,FasterAI: Prune and Distill your models with FastAI and PyTorch
User: nathanhubens
Home Page: https://nathanhubens.github.io/fasterai/
pruning compression fastai pytorch knowledge-distillation

open-mmlab / mmrazor
knowledge-distillation,OpenMMLab Model Compression Toolbox and Benchmark.
Organization: open-mmlab
Home Page: https://mmrazor.readthedocs.io/en/latest/
nas pruning knowledge-distillation spos darts autoslim detection segmentation classification pytorch

paddlepaddle / paddleclas
knowledge-distillation,A treasure chest for visual classification and recognition powered by PaddlePaddle
Organization: paddlepaddle
image-classification knowledge-distillation autoaugment randaugment gridmask cutmix deit repvgg swin-transformer pretrained-models

picsart-ai-research / mi-gan
knowledge-distillation,[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices
Organization: picsart-ai-research
image-inpainting knowledge-distillation object-removal mi-gan mobile-inpainting re-parameterization

sebastian-hofstaetter / matchmaker
knowledge-distillation,Training & evaluation library for text-based neural re-ranking and dense retrieval models built with PyTorch
User: sebastian-hofstaetter
Home Page: https://neural-ir-explorer.ec.tuwien.ac.at/
neural-ir transformer explainability efficiency knowledge-distillation dpr pytorch neural-ranking

segmind / distill-sd
knowledge-distillation,Segmind Distilled diffusion
Organization: segmind
Home Page: https://discord.gg/p2MdJqZXnb
distillation inference knowledge-distillation stable-diffusion

sforaidl / kd_lib
knowledge-distillation,A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.
Organization: sforaidl
Home Page: https://kd-lib.readthedocs.io/
algorithm-implementations benchmarking data-science deep-learning-library knowledge-distillation machine-learning model-compression pruning pytorch quantization

sseung0703 / kd_methods_with_tf
knowledge-distillation,Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)
User: sseung0703
knowledge-distillation tensorflow-examples tensorflow

ssisyphustao / object-detection-knowledge-distillation
knowledge-distillation,An Object Detection Knowledge Distillation framework powered by pytorch, now having SSD and yolov5.
User: ssisyphustao
pytorch knowledge-distillation ssd object-detection deep-learning yolov5

szagoruyko / attention-transfer
knowledge-distillation,Improving Convolutional Networks via Attention Transfer (ICLR 2017)
User: szagoruyko
Home Page: https://arxiv.org/abs/1612.03928
attention deep-learning knowledge-distillation pytorch

tebmer / awesome-knowledge-distillation-of-llms
knowledge-distillation,This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.
User: tebmer
data-augmentation instruction-following kd knowledge-distillation large-language-model llm self-training survey compression data-synthesis

vita-group / slak
knowledge-distillation,[ICLR 2023] "More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity"; [ICML 2023] "Are Large Kernels Better Teachers than Transformers for ConvNets?"
Organization: vita-group
convnet large-kernels object-detection pytorch segmentation 51x51 deep-learning dynamic-sparsity knowledge-distillation

yanx27 / 2dpass
knowledge-distillation,2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (ECCV 2022) :fire:
User: yanx27
nuscenes semantic-segmentation semantickitti lidar-point-cloud deep-learning knowledge-distillation pytorch

yjiangcm / lion
knowledge-distillation,Code for "Lion: Adversarial Distillation of Proprietary Large Language Models (EMNLP 2023)"
User: yjiangcm
Home Page: https://arxiv.org/abs/2305.12870
instruction-following knowledge-distillation large-language-models

yoshitomo-matsubara / torchdistill
knowledge-distillation,A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
User: yoshitomo-matsubara
Home Page: https://yoshitomo-matsubara.net/torchdistill/
knowledge-distillation pytorch image-classification imagenet object-detection coco semantic-segmentation cifar10 cifar100 colab-notebook

yuanli2333 / teacher-free-knowledge-distillation
knowledge-distillation,Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
User: yuanli2333
knowledge-distillation pytorch teacher-free paper-implementations label-smoothing

yuanyuanli85 / fast_human_pose_estimation_pytorch
knowledge-distillation,Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
User: yuanyuanli85
pytorch knowledge-distillation stacked-hourglass-networks pose-estimation

yzd-v / fgd
knowledge-distillation,Focal and Global Knowledge Distillation for Detectors (CVPR 2022)
User: yzd-v
deep-learning knowledge-distillation object-detection pytorch

yzd-v / mgd
knowledge-distillation,Masked Generative Distillation (ECCV 2022)
User: yzd-v
image-classification instance-segmentation knowledge-distillation object-detection pytorch semantic-segmentation

yzhao062 / suod
knowledge-distillation,(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)
User: yzhao062
Home Page: https://www.andrew.cmu.edu/user/yuezhao2/papers/20-preprint-suod.pdf
data-mining machine-learning anomaly-detection outlier-detection distributed-systems knowledge-distillation python machine-learning-algorithms machine-learning-library