CLRNet-onnxruntime-and-tensorrt-demo

This is the onnxruntime and tensorrt inference code for CLRNet: Cross Layer Refinement Network for Lane Detection (CVPR 2022). Official code: https://github.com/Turoad/CLRNet

skip to onnx demo and skip to tensorrt demo

Note

1、Making onnx supported op grid_sampler.
2、Using this code you can successfully convert to onnx model and inference an onnxruntime demo. A new version demo only use numpy to do post-processing, easy to deploy but more time cost for NMS.
3、Modifications according to the following operations will affect the training code, this code only for onnx inference.
4、It mainly includes two parts: model inference and post-processing.
5、Supporting convert to tensorrt engine.

convert and test onnx

1、git official code and install original environment by refer to https://github.com/Turoad/CLRNet
2、git clone this code
3、cp clr_head.py to your_path/CLRNet/clrnet/models/heads/
4、mkdir your_path/CLRNet/modules/ and cp grid_sample.py to your_path/CLRNet/modules/
5、cp torch2onnx.py to your_path/CLRNet/
6、For example, run

python torch2onnx.py configs/clrnet/clr_resnet18_tusimple.py  --load_from tusimple_r18.pth

my deployment log is here https://github.com/xuanandsix/CLRNet-onnxruntime-and-tensorrt-demo/blob/main/my_log/test_onnx.log
7、cp test.jpg to your_path/CLRNet/ and run

1. NMS based on torch and cpython.

python demo_onnx.py

2. NMS based on numpy.

python demo_onnx_new.py 

onnx output

convert and test tensorrt

Tensorrt version needs to be greater than 8.4. This code is implemented in TensorRT-8.4.0.6.
GatherElements error、IShuffleLayer error、`is_tensor()' failed have been resolved.
1、install tensorrt and compilation tools trtexec.
2、install polygraphy to help modify the onnx model. You can install by

pip install nvidia-pyindex
pip install polygraphy
pip install onnx-graphsurgeon

and run

polygraphy surgeon sanitize your_path/tusimple_r18.onnx --fold-constants --output your_path/tusimple_r18.onnx

3、 convert to tensorrt and get tusimple_r18.engine

./trtexec --onnx=your_path/tusimple_r18.onnx --saveEngine=your_path/tusimple_r18.engine --verbose

4、test tensorrt demo.

python demo_trt.py

tensorrt output

TO DO

• Optimize post-processing.
• Tensorrt demo.
• Cpp demo.