Radxa Model-Zoo is based on the Sophon demo and has been further developed independently by Radxa. Radxa Model-Zoo provides a collection of ported routines and precompiled TPU platform models for mainstream algorithms. This includes model compilation and quantization based on TPU-NNTC and TPU-MLIR, porting of the inference engine based on BMRuntime, and algorithm porting for pre- and post-processing based on BMCV/OpenCV.
Radxa Model-Zoo is developed using the SophonSDK interface. SophonSDK is a deep learning SDK customized by Algorithmic Capability Technology based on its independently developed deep learning processor. It covers capabilities such as model optimization and efficient runtime support needed in the neural network inference stage, providing an easy-to-use, efficient full-stack solution for deep learning application development and deployment. Currently, it is compatible with BM1684X.
| Directory | Algorithm Category | Programming Language | BModel | Multi-Batch Support | Preprocessing Library |
|---|---|---|---|---|---|
| LPRNet | License Plate Recognition | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| ResNet | Image Classification | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| RetinaFace | Face Detection | C++/Python | FP32 | YES | BMCV/OpenCV |
| segformer | Semantic Segmentation | C++/Python | FP32/FP16 | YES | BMCV/OpenCV |
| yolact | Instance Segmentation | Python | FP32 | YES | BMCV/OpenCV |
| PP-OCR | OCR | C++/Python | FP32/FP16 | YES | BMCV/OpenCV |
| OpenPose | Human Pose Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| C3D | Video Action Recognition | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| DeepSORT | Multi-Object Tracking | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| ByteTrack | Multi-Object Tracking | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| CenterNet | Object Detection, Pose Recognition | C++/Python | FP32/FP16/INT8 | YES | BMCV |
| YOLOv5 | Object Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| YOLOv34 | Object Detection | C++/Python | FP32/INT8 | NO | BMCV/OpenCV |
| YOLOX | Object Detection | C++/Python | FP32/INT8 | YES | BMCV/OpenCV |
| SSD | Object Detection | C++/Python | FP32/INT8 | YES | BMCV/OpenCV |
| YOLOv7 | Object Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| YOLOv8 | Object Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| YOLOv5_opt | Object Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| ppYOLOv3 | Object Detection | C++/Python | FP32/FP16/INT8 | YES | BMCV/OpenCV |
| WeNet | Speech Recognition | C++/Python | FP32/FP16 | YES | - |
| BERT | Language Model | C++/Python | FP32/FP16 | YES | - |
| ChatGLM2 | Language Model | C++/Python | FP16/INT8/INT4 | YES | - |
| Llama2 | Language Model | C++ | FP16/INT8/INT4 | YES | - |
Radxa Model-Zoo primarily depends on tpu-mlir, tpu-nntc, libsophon, sophon-ffmpeg, sophon-opencv, and sophon-sail. The version requirements are as follows:
| Radxa Model-Zoo | tpu-mlir | tpu-nntc | libsophon | sophon-ffmpeg | sophon-opencv | sophon-sail | Release Date |
|---|---|---|---|---|---|---|---|
| 0.1.0 | >=1.2.2 | >=3.1.7 | >=0.4.6 | >=0.6.0 | >=0.6.0 | >=3.7.0 | >=23.10.01 |