Triton Inference Server

[hongbo-miao]

Triton Inference Server

• https://github.com/triton-inference-server/server

Guide

• https://github.com/triton-inference-server/tutorials#quick-deploy
• https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/getting_started/quickstart.html

Triton Inference Server on SageMaker

• https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-triton/resnet50/triton_resnet50.ipynb
• SageMaker Multi-Model Endpoint (MME): https://github.com/aws/amazon-sagemaker-examples/blob/main/multi-model-endpoints/mme-on-gpu/cv/resnet50_mme_with_gpu.ipynb

[hongbo-miao]

Added by

• feat(machine-learning): add triton inference server #16849
• feat(triton): add triton client #16918
• https://github.com/hongbo-miao/hongbomiao.com/pull/16930