lessw2020 Goto Github PK

4bit_optimizers

4bit optimizers for PyTorch

5d-compiler

Auto-Parallelization Compiler using 4D Parallel + Checkpointing (5D)

actnn

ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

adair

AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

adan

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

advanced-nlp

Implementation of the latest developments in NLP ranging from ensemble learning to BERT. Pipelines are in Tensorflow and Pytorch

ai_helpers

collection of various utility functions for making model training easier

alcop-artifact

algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

alibi_positional_embeddings

Alibi in PyTorch

alicemind

ALIbaba's Collection of Encoder-decoders from MinD (Machine IntelligeNce of Damo) Lab

aot_fsdp

When AOT Autograd meets FSDP = large models train faster

asynch-checkpointing

PyTorch checkpointing

atom

Codebase for Atom

auto-adaptive-ai

auto adaptive framework for intrinsic hyperparameter selection, adaptive padding, normalized weights

autogptq_triton

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

autoopt-for-fastai

Integrate Ebay's AutoOpt Deep Learning Optimizer into the FastAI framework

best-deep-learning-optimizers

Collection of the latest, greatest, deep learning optimizers (for Pytorch) - CNN, NLP suitable

bfloat_optimizer

Pure bfloat AdamW+ tweaks

bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

came_optimizer

The official implementation of "CAME: Confidence-guided Adaptive Memory Optimization"

cartpole-cross-entropy-agent

Cross entropy reinforcement learning agent for solving Cartpole in OpenAI Gym.

cfx-research

comm_fusion

fork of Shen's communication fusion repo

coordinate_clipped_optimizers

coordinate wise clipped Optimizers in PyTorch

cswin-transformer

CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped

cuda-kernel-dev

in progress cuda kernels

cuda-olcf

Training materials associated with NVIDIA's CUDA Training Series (www.olcf.ornl.gov/cuda-training-series/)

cuda-techniques

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

curriculum-learning-dropout

Implementation of Curriculum Learning Dropout for FastAI framework