torchpipe / torchpipe Goto Github PK
View Code? Open in Web Editor NEWAn Alternative for Triton Inference Server. Boosting DL Service Throughput 1.5-4x by Ensemble Pipeline Serving with Concurrent CUDA Streams for PyTorch/LibTorch Frontend and TensorRT/CVCUDA, etc., Backends
Home Page: https://torchpipe.github.io/
License: Apache License 2.0