T4 and TensorRT Survey

t4tensortsurvey.md

Prepare

reference: https://github.com/NVIDIA/retinanet-examples

CLI session dump

if you see retinanet_rn50fpn.pth crashed, please see solution here

root@4f884f99f02d:~/work# odtk export retinanet_rn50fpn.pth  retinanet_rn50fpn.plan
Loading model from retinanet_rn50fpn.pth...
     model: RetinaNet
  backbone: ResNet50FPN
   classes: 80, anchors: 9
Bus error (core dumped)

BTW, mobilenet.pth works

root@4f884f99f02d:~/work# odtk export mobilenet.pth mobilenet.plan
Loading model from mobilenet.pth...
     model: RetinaNet
  backbone: MobileNetV2FPN
   classes: 80, anchors: 9
Exporting to ONNX...
Building FP16 core model...
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...
Writing to mobilenet.plan...
[E ProcessGroupNCCL.cpp:284] NCCL watchdog thread terminated

• Download and extract pre-trained retinanet model from the 19.04 release

wget https://github.com/NVIDIA/retinanet-examples/releases/download/19.04/retinanet_rn50fpn.zip
unzip retinanet_rn50fpn.zip 
Archive:  retinanet_rn50fpn.zip
  inflating: retinanet_rn50fpn/retinanet_rn50fpn.pth 

• if everything is fine

root@864c38b7c156:~/retailbase/work# odtk export  retinanet_rn50fpn.pth  retinanet_rn50fpn.engine
Loading model from retinanet_rn50fpn.pth...
     model: RetinaNet
  backbone: ResNet50FPN
   classes: 80, anchors: 9
Exporting to ONNX...
Building FP16 core model...
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...
Writing to retinanet_rn50fpn.engine...
[E ProcessGroupNCCL.cpp:284] NCCL watchdog thread terminated