⚗️ Instill Model manages the AI model-related resources and features working with Instill VDP.

☁️ Instill Cloud offers fully managed Instill Model. Please sign up to try out for free.

• ⚡️ High-performing inference implemented in Go with Triton Inference Server for unleashing the full power of NVIDIA GPU architecture (e.g., concurrency, scheduler, batcher) supporting TensorRT, PyTorch, TensorFlow, ONNX, Python and more.

• 🖱️ One-click model deployment from GitHub, Hugging Face or cloud storage managed by version control tools like DVC or ArtiVC.

• 📦 Standardised AI Task output formats to streamline data integration or analysis

• macOS or Linux - VDP works on macOS or Linux, but does not support Windows yet.

• Docker and Docker Compose - VDP uses Docker Compose (specifically, Compose V2 and Compose specification) to run all services at local. Please install the latest stable Docker and Docker Compose before using VDP.

• yq > v4.x. Please follow the installation guide.

• (Optional) NVIDIA Container Toolkit - To enable GPU support in VDP, please refer to NVIDIA Cloud Native Documentation to install NVIDIA Container Toolkit. If you'd like to specifically allot GPUs to VDP, you can set the environment variable NVIDIA_VISIBLE_DEVICES. For example, NVIDIA_VISIBLE_DEVICES=0,1 will make the triton-server consume GPU device id 0 and 1 specifically. By default NVIDIA_VISIBLE_DEVICES is set to all to use all available GPUs on the machine.

Execute the following commands to start pre-built images with all the dependencies:

The stable release version

$ git clone -b v0.4.0-alpha https://github.com/instill-ai/model.git && cd model

# Launch all services
$ make all

The latest version for development

$ git clone https://github.com/instill-ai/model.git && cd model

# Launch all services
$ make latest PROFILE=all

Note Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.

Note The image of model-backend (~2GB) and Triton Inference Server (~23GB) can take a while to pull, but this should be an one-time effort at the first setup.

We curate a list of ready-to-use models. These pre-trained models are from different sources and have been trained and deployed by our team. Want to contribute a new model? Please create an issue, we are happy to add it to the list 👐.

Note: The GitHub-DVC source in the table means importing a model into VDP from a GitHub repository that uses DVC to manage large files.

📔 Documentation

Please check out the documentation website.

📘 API Reference

The gRPC protocols in protobufs provide the single source of truth for the VDP APIs. The genuine protobuf documentation can be found in our Buf Scheme Registry (BSR).

For the OpenAPI documentation, access http://localhost:3001 after make all, or simply run make doc.

Please refer to the Contributing Guidelines for more details.

Please refer to the community repository.

See the LICENSE file for licensing information.