Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

Add bisenetv2. My implementation of BiSeNet

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server

OpenAI compatible API for TensorRT LLM triton backend

【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接口方式提供服务。

Serving Inside Pytorch

ClearML - Model-Serving Orchestration and Repository Solution

The Triton backend for the ONNX Runtime.

Deploy stable diffusion model with onnx/tenorrt + tritonserver

NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU

Deploy DL/ ML inference pipelines with minimal extra code.

Анализ трафика на круговом движении с использованием компьютерного зрения

Compare multiple optimization methods on triton to imporve model service performance

Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.

Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.

Tiny configuration for Triton Inference Server

Diffusion Model for Voice Conversion

Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX

Provides an ensemble model to deploy a YoloV8 ONNX model to Triton

triton server ensemble model demo