ztaoplus Goto Github PK

argoexec

argoexec:

candle

Minimalist ML framework for Rust

ci-pipeline

ci-pipeline

codebox-api

CodeBox is the simplest cloud infrastructure for your LLM Apps and Services.

codeinterpreter-api

Open source implementation of the ChatGPT Code Interpreter 👾

container-images

Common container images

enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

flash-attention

Fast and memory-efficient exact attention

guidance

A guidance language for controlling large language models.

image-mirror

mirror unreachable images

inference-framework-benchmark

Benchmark for various inference frameworks

jkpmanager

jupyter kernels pool manager

jupyter-images

Kubeflow Jupyter images

langchain

🦜🔗 Build context-aware reasoning applications

litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

llama-efficient-tuning

Easy-to-use fine-tuning framework using PEFT (PT+SFT+RLHF with QLoRA) (LLaMA-2, BLOOM, Falcon, Baichuan)

llm-awq

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

lmdeploy

lmdeploy copy from docker io

mindsdb

MindsDB connects AI models to real time data

mirrored-image

qizhenmedicalexpert

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

sglang

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

tensorrt-llm

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

tensorrtllm_backend

The Triton TensorRT-LLM Backend

text-generation-inference

Large Language Model Text Generation Inference

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

ztaoplus

I know you know what I mean..