v0.3.0 Linux GPU: TensorRT

New Features: NVIDIA GPU Inference support via TensorRT

🎉🎉TensorRT: Boost inference performance with NVIDIA GPU via TensorRT.

Run bash ./build.sh tensorrt to build lite.ai.toolkit with TensorRT support, and then test yolov5 with the codes below. NOTE: lite.ai.toolkit need TensorRT 10.x (or later) and CUDA 12.x (or later). Please check build.sh, tensorrt-linux-x86_64-install.zh.md, test_lite_yolov5.cpp and NVIDIA/TensorRT for more details.

// trtexec --onnx=yolov5s.onnx --saveEngine=yolov5s.engine
auto *yolov5 = new lite::trt::cv::detection::YOLOV5(engine_path);

Class	Class	Class	Class	Class	System	Engine
✅YOLOv5	✅YOLOv6	✅YOLOv8	✅YOLOv8Face	✅YOLOv5Face	Linux	TensorRT
✅YOLOX	✅YOLOv5BlazeFace	✅StableDiffusion	/	/	Linux	TensorRT

What's Changed

• Add Yoloface-8n by @wangzijian1010 in #412
• [README] Update authors by @DefTruth in #413
• 增加Linux下的TensorRT的安装文档 by @wangzijian1010 in #415
• update first tensortt inference example by @wangzijian1010 in #416
• Update TensorRT BasicHandler and add yolov5 cpp file by @wangzijian1010 in #417
• add yolox implement by @wangzijian1010 in #418
• [TensorRT] support YOLOv8 with TensorRT backend by @wangzijian1010 in #419
• [TensorRT] support YOLOv6 with TensorRT backend by @wangzijian1010 in #420
• [TensorRT] support YOLOV5Face with TensorRT backend by @wangzijian1010 in #421
• [TensorRT] fix undefine bug in yolov5face trt version by @wangzijian1010 in #422
• [onnxruntime] Clip onnx update by @wangzijian1010 in #424
• [Misc] Remove un-need comments by @wangzijian1010 in #425
• [TensorRT] Clip TensorRT code update by @wangzijian1010 in #426
• Update README.md by @DefTruth in #430
• [TensorRT] [ORT] support Stable Diffusion text2img Pipeline by @wangzijian1010 in #432
• [Bugfix] Fix some build errors for SD by @DefTruth in #434
• [Bugfix] Fix OrtCompatiableGetInputName errors by @DefTruth in #435
• [Fix Bug] remove duplicate code by @wangzijian1010 in #436
• Bump up to v0.3.0 by @DefTruth in #438

New Contributors

• @wangzijian1010 made their first contribution in #412

Full Changelog: v0.2.0...v0.3.0-rc1

v0.2.0 Linux CPU: ONNXRuntime + MNN

Full Changelog: v0.2.0-rc3...v0.2.0

v0.2.0-rc3

Full Changelog: v0.2.0-rc2...v0.2.0-rc3

v0.2.0-rc2

Full Changelog: v0.2.0-rc1...v0.2.0-rc2

v0.2.0-rc1: Merge pull request #399 from DefTruth/linux-dev

Bump to 0.2.0+ort1.17.1+ocv4.9.0

v0.2.0-rc0

pre-release

Lite Ai Toolkit Mac/Linux v0.1.1

v0.1.1 release notes

预编译计划 MacOS系统 v0.1.1版本 (prebuilt plan)

• 2022/02/29 MacOS下的lite.ai.toolkit预编译库
  • lite0.1.1-osx10.15.x-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.7.0.zip
  • lite0.1.1-osx10.15.x-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.8.1.zip
  • lite0.1.1-osx10.15.x-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.9.0.zip
  • lite0.1.1-osx10.15.x-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.10.0.zip
  • lite0.1.1-osx10.15.x-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.7.0-mnn1.2.0-ncnn1.0.21-tnn0.3.0.zip

预编译计划 Linux系统 v0.1.1版本 (prebuilt plan)

• 2022/03/07 Linux下的lite.ai.toolkit预编译库
  • lite0.1.1-ubuntu18.04-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.7.0.zip
  • lite0.1.1-ubuntu18.04-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.8.1.zip
  • lite0.1.1-ubuntu18.04-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.9.0.zip
  • lite0.1.1-ubuntu18.04-ocv4.5.2-ffmpeg4.2.2-onnxruntime1.10.0.zip

lite.ai.toolkit-v0.1.0

lite.ai.toolkit-v0.1.0: Release the source code of Lite.AI.ToolKit . It contains almost 70+ AI models and re-implements with onnxruntime c++ by default.