disco-diffusion - A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.
midjourney - Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.
segment-anything (SAM) - produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image.
bark - Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects.
whisper - Robust Speech Recognition via Large-Scale Weak Supervision
Deepchecks - Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
Evidently - Evaluate and monitor ML models from validation to production.
Arize-Phoenix - ML observability for LLMs, vision, language, and tabular models.
deeplake - Stream large multimodal datasets to achieve near 100% GPU utilization. Query, visualize, & version control data. Access data w/o the need to recompute the embeddings for the model finetuning.
GPTCache - Creating semantic cache to store responses from LLM queries.
Haystack - Quickly compose applications with LLM Agents, semantic search, question-answering and more.
langchain - Building applications with LLMs through composability
LlamaIndex - Provides a central interface to connect your LLMs with external data.
xTuring - Build and control your personal LLMs with fast and efficient fine-tuning.
ZenML - Open-source framework for orchestrating, experimenting and deploying production-grade ML solutions, with built-in langchain & llama_index integrations.
Milvus - Vector database for scalable similarity search and AI applications.
Pinecone - The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles
Qdrant - Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud
Vald - A Highly Scalable Distributed Vector Search Engine
Vearch - A distributed system for embedding-based vector retrieval
Weaviate - Weaviate is an open source vector search engine that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients.
ColossalAI - An integrated large-scale model training system with efficient parallelization techniques.
DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
Jax - Autograd and XLA for high-performance machine learning research.
Kedro - Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
Keras - Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow.
LightGBM - A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
MegEngine - MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto-differentiation.
metric-learn - Metric Learning Algorithms in Python.
MindSpore - MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
Oneflow - OneFlow is a performance-centered and open-source deep learning framework.
PaddlePaddle - Machine Learning Framework from Industrial Practice.
PyTorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration.
PyTorchLightning - The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
XGBoost - Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library.
Kedro-Viz - Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project.
LabNotebook - LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments.
Sacred - Sacred is a tool to help you configure, organize, log and reproduce experiments.
TensorSpace - Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js.
dtreeviz - A python library for decision tree visualization and model interpretation.
Zetane Viewer - ML models and internal tensors 3D visualizer.
Zeno - AI evaluation platform for interactively exploring data and model outputs.
ArtiVC - A version control system to manage large files.
Lake is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size.
Piperider - A CLI tool that allows you to build data profiles and write assertion tests for easily evaluating and tracking your data's reliability over time.
LUX - A Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process.
Upgini - Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features
Feast - An open source feature store for machine learning.
ONNX-MLIR - Compiler technology to transform a valid Open Neural Network Exchange (ONNX) graph into code that implements the graph with minimum runtime support.
TVM - Open deep learning compiler stack for cpu, gpu and specialized accelerators
octoml-profile - octoml-profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware with state-of-the-art ML acceleration technology.
scalene - a high-performance, high-precision CPU, GPU, and memory profiler for Python
Auto-PyTorch - Automatic architecture search and hyperparameter optimization for PyTorch.
auto-sklearn - an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
Dragonfly - An open source python library for scalable Bayesian optimisation.
Determined - scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods.
DEvol (DeepEvolution) - a basic proof of concept for genetic architecture search in Keras.
EvalML - An open source python library for AutoML.
FEDOT - AutoML framework for the design of composite pipelines.
learn2learn - PyTorch Meta-learning Framework for Researchers.
Ludwig - a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code.
MOE - a global, black box optimization engine for real world metric optimization by Yelp.
Model Search - a framework that implements AutoML algorithms for model architecture search at scale.
NASGym - a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS).
NNI - An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
TensorFlow Model Optimization - A suite of tools that users, both novice and advanced, can use to optimize machine learning models for deployment and execution.
TNN - A uniform deep learning inference framework for mobile, desktop and server.
EasyFL - An Easy-to-use Federated Learning Platform
FATE - An Industrial Grade Federated Learning Framework
FedML - The federated learning and analytics library enabling secure and collaborative machine learning on decentralized data anywhere at any scale. Supporting large-scale cross-silo federated learning, cross-device federated learning on smartphones/IoTs, and research simulation.
Harmonia - Harmonia is an open-source project aiming at developing systems/infrastructures and libraries to ease the adoption of federated learning (abbreviated to FL) for researches and production usage.
visenger/awesome-mlops - An awesome list of references for MLOps - Machine Learning Operations
currentslab/awesome-vector-search - A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search.
- Code and documentation to train Stanford's Alpaca models, and generate the data.
- A 7B Large Language Model fine-tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca.
- BigScience Large Open-science Open-access Multilingual Language Model
- Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform
- An Open Bilingual Pre-Trained Model, quantization of ChatGLM-130B, can run on consumer-level GPUs.
- An Open Bilingual Pre-Trained Model (ICLR 2023)
- An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
- A Chinese LLM, Based on LLaMA and fine tune by Stanford Alpaca, Alpaca LoRA, Japanese-Alpaca-LoRA.
- StableLM: Stability AI Language Models
- A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.
- produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image.
- A latent text-to-image diffusion model
- High-Resolution Image Synthesis with Latent Diffusion Models
- Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects.
- Robust Speech Recognition via Large-Scale Weak Supervision
- Alpaca-LoRA as Chatbot service
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
- Running large language models on a single GPU for throughput-oriented scenarios.
- Drag & drop UI to build your customized LLM flow using LangchainJS.
- Port of Facebook's LLaMA model in C/C++
- Port of OpenAI's Whisper model in C/C++
- Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention.
- The Unified Model Serving Framework
- A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface.
- A flexible, high-performance serving system for machine learning models.
- Serve, optimize and scale PyTorch models in production
- The Triton Inference Server provides an optimized cloud and edge inferencing solution.
- Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
- Evaluate and monitor ML models from validation to production.
- Always know what to expect from your data.
- The open standard for data logging
- ML observability for LLMs, vision, language, and tabular models.
- Stream large multimodal datasets to achieve near 100% GPU utilization. Query, visualize, & version control data. Access data w/o the need to recompute the embeddings for the model finetuning.
- Creating semantic cache to store responses from LLM queries.
- Quickly compose applications with LLM Agents, semantic search, question-answering and more.
- Building applications with LLMs through composability
- Provides a central interface to connect your LLMs with external data.
- Build and control your personal LLMs with fast and efficient fine-tuning.
- Open-source framework for orchestrating, experimenting and deploying production-grade ML solutions, with built-in
- An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search.
- the open source embedding database
- Build multimodal AI services via cloud native technologies · Neural Search · Generative AI · Cloud Native
- Tensor search for humans.
- Vector database for scalable similarity search and AI applications.
- Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud
- Build AI-powered semantic search applications
- A Highly Scalable Distributed Vector Search Engine
- A distributed system for embedding-based vector retrieval
- Weaviate is an open source vector search engine that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients.
- CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.
- An open-source alternative to GitHub Copilot server
- Self-hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot.
- Run VS Code on any machine anywhere and access it in the browser.
- OS-agnostic, system-level binary package manager and ecosystem.
- Moby is an open-source project created by Docker to enable and accelerate software containerization.
- 🏕️ Reproducible development environment for AI/ML.
- The Jupyter notebook is a web-based notebook environment for interactive computing.
- A build, packaging, and run system for ephemeral multi-container environments.
- Instruct-tune LLaMA on consumer hardware
- An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
- Using Low-rank adaptation to quickly fine-tune diffusion models.
- State-of-the-art Parameter-Efficient Fine-Tuning.
- An optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. (ACL 2022)
- 🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision.
- Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler.
- A fast open framework for deep learning.
- An integrated large-scale model training system with efficient parallelization techniques.
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
- Autograd and XLA for high-performance machine learning research.
- Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
- Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow.
- A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
- MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto-differentiation.
- Metric Learning Algorithms in Python.
- MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
- OneFlow is a performance-centered and open-source deep learning framework.
- Machine Learning Framework from Industrial Practice.
- Tensors and Dynamic neural networks in Python with strong GPU acceleration.
- The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
- Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library.
- Machine Learning in Python.
- An Open Source Machine Learning Framework for Everyone.
- A minimalist neural network library optimized for sparse data and single machine environments.
- an easy-to-use and performant open-source experiment tracker.
- Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management
- Experiment tracking, ML developer tools.
- Machine Learning automation and tracking.
- Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project.
- LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments.
- Sacred is a tool to help you configure, organize, log and reproduce experiments.
- A model-agnostic visual debugging tool for machine learning.
- Visualizer for neural network, deep learning, and machine learning models.
- Bring multiple data streams into one dashboard.
- TensorFlow's Visualization Toolkit.
- Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js.
- A python library for decision tree visualization and model interpretation.
- ML models and internal tensors 3D visualizer.
- AI evaluation platform for interactively exploring data and model outputs.
- A version control system to manage large files. Lake is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size.
- Git for Data.
- Data Version Control | Git for Data & Models | ML Experiments Management.
- Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
- Pachyderm is a version control system for data.
- A self-organizing data hub for S3.
- A distributed POSIX file system built on top of Redis and S3.
- Git-like capabilities for your object storage.
- Modern columnar data format for ML implemented in Rust.
- A CLI tool that allows you to build data profiles and write assertion tests for easily evaluating and tracking your data's reliability over time.
- A Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process.
- The Virtual Feature Store. Turn your existing data infrastructure into a feature store.
- An open source python framework for automated feature engineering
- Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features
- An open source feature store for machine learning.
- Open source platform for the machine learning lifecycle.
- ModelFox is a platform for managing and deploying machine learning models.
- Standardized Serverless ML Inference Platform on Kubernetes
- Machine Learning Toolkit for Kubernetes.
- Resource scheduling and cluster management for AI.
- Machine Learning Management & Orchestration Platform.
- An effortless infrastructure for machine learning built on the top of Kubernetes.
- An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
- A platform to programmatically author, schedule and monitor workflows.
- An Open-Source Platform for Production Data Science
- Workflow engine for Kubernetes.
- Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale.
- Machine Learning Pipelines for Kubeflow.
- Build and manage real-life data science projects with ease!
- The fastest way to build data pipelines. Develop iteratively, deploy anywhere.
- The easiest way to automate your data.
- An open-source unstructured data ETL tool to streamline the end-to-end unstructured data processing pipeline.
- Kubernetes-native Job Queueing.
- A Highly Scalable Workload Manager.
- A Cloud Native Batch System (Project under CNCF).
- Light-weight, universal resource scheduler for container orchestrator systems.
- Open Source ML Model Versioning, Metadata, and Experiment Management
- A tool to package, serve, and deploy any ML model on any platform.
- Docker for Your ML/DL Models Based on OCI Artifacts
- Compiler technology to transform a valid Open Neural Network Exchange (ONNX) graph into code that implements the graph with minimum runtime support.
- Open deep learning compiler stack for cpu, gpu and specialized accelerators
- octoml-profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware with state-of-the-art ML acceleration technology.
- a high-performance, high-precision CPU, GPU, and memory profiler for Python
- a platform for Neural Network Search (NAS) that allows you to generate efficient deep networks for your applications.
- A framework to find the best performing AI/ML model for any AI problem.
- An autoML framework & toolkit for machine learning on graphs
- AutoML for Image, Text, and Tabular Data.
- Provide an input CSV and a target field to predict, generate a model + code to run it.
- AutoML library for deep learning.
- Automatic architecture search and hyperparameter optimization for PyTorch.
- an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
- An open source python library for scalable Bayesian optimisation.
- scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods.
- a basic proof of concept for genetic architecture search in Keras.
- An open source python library for AutoML.
- AutoML framework for the design of composite pipelines.
- Fast and lightweight AutoML (paper).
- A hyperparameter optimization framework, inspired by Optuna.
- a framework for distributed hyperparameter optimization.
- a library for hyperparameter optimization and black box optimization benchmarks.
- open source code for tuning hyperparams with Hyperband.
- A General Automated Machine Learning Framework.
- Distributed Asynchronous Hyperparameter Optimization in Python.
- A toolset for black-box hyperparameter optimisation.
- Katib is a Kubernetes-native project for automated machine learning (AutoML).
- Hyperparameter tuning for humans.
- PyTorch Meta-learning Framework for Researchers.
- a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code.
- a global, black box optimization engine for real world metric optimization by Yelp.
- a framework that implements AutoML algorithms for model architecture search at scale.
- a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS).
- An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
- A hyperparameter optimization framework.
- An open-source, low-code machine learning library in Python that automates machine learning workflows.
- Scalable Hyperparameter Tuning.
- Bayesian optimization in high-dimensions via random embedding.
- a Robust Bayesian Optimization framework.
- Sequential model-based optimization with a
- a software package to perform Bayesian optimization.
- one of the very first AutoML methods and open-source software packages.
- A Meta-Learning library for PyTorch.
- an AutoML algorithm tool chain by Huawei Noah's Arb Lab.
- FeatherCNN is a high performance inference engine for convolutional neural networks.
- A library for high performance deep learning inference on NVIDIA GPUs.
- ncnn is a high-performance neural network inference framework optimized for the mobile platform.
- use AutoML to do model compression.
- A suite of tools that users, both novice and advanced, can use to optimize machine learning models for deployment and execution.
- A uniform deep learning inference framework for mobile, desktop and server.
- An Easy-to-use Federated Learning Platform
- An Industrial Grade Federated Learning Framework
- The federated learning and analytics library enabling secure and collaborative machine learning on decentralized data anywhere at any scale. Supporting large-scale cross-silo federated learning, cross-device federated learning on smartphones/IoTs, and research simulation.
- A Friendly Federated Learning Framework
- Harmonia is an open-source project aiming at developing systems/infrastructures and libraries to ease the adoption of federated learning (abbreviated to FL) for researches and production usage.
- A framework for implementing federated learning
- A curated list of awesome projects and resources related to Argo
- Automated Deep Learning: Neural Architecture Search Is Not the End (a curated list of AutoDL resources and an in-depth analysis)
- Curating a list of AutoML-related research, tools, projects and other resources
- A curated list of automated machine learning papers, articles, tutorials, slides and projects
- A curated list of federated learning publications, re-organized from Arxiv (mostly)
- All materials you need for Federated Learning: blogs, videos, papers, and softwares, etc.
- This is the Fuzzy Labs guide to the universe of free and open source MLOps tools.
- A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
- A list of awesome compiler projects and papers for tensor computation and deep learning.
- A curated list of awesome MLOps tools.
- An awesome list of references for MLOps - Machine Learning Operations
- A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent