# trtexec ( nv test tool )
https://github.com/NVIDIA/TensorRT/tree/release/8.6/samples/trtexec

trtexec is a tool to quickly utilize TensorRT without having to develop your own application. The trtexec tool has two main purposes:

* It’s useful for benchmarking networks on random or user-provided input data.
* It’s useful for generating serialized engines from models.

**Benchmarking network** - If you have a model saved as a UFF file, ONNX file, or if you have a network description in a Caffe prototxt format, you can use the trtexec tool to test the performance of running inference on your network using TensorRT. The trtexec tool has many options for specifying inputs and outputs, iterations for performance timing, precision allowed, and other options.

**Serialized engine generation** - If you generate a saved serialized engine file, you can pull it into another application that runs inference. For example, you can use the TensorRT Laboratory to run the engine with multiple execution contexts from multiple threads in a fully pipelined asynchronous way to test parallel inference performance. There are some caveats, for example, if you used a Caffe prototxt file and a model is not supplied, random weights are generated. Also, in INT8 mode, random weights are used, meaning trtexec does not provide calibration capability.

# Models

測試模型與出處
測試參數來源
https://developer.nvidia.com/zh-cn/blog/tensorrt-trtexec-cn/

## How To

PeopleSemSegFormer 模型名稱:peoplesemsegformer.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplesemsegformer

指令語法:
* trtexec --onnx=peoplesemsegformer.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=peoplesemsegformer.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=peoplesemsegformer.engine --batch=2

Retail Object Detection 模型名稱:retail_object_detection_dino_meta.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/retail_object_detection/version

指令語法:
* trtexec --onnx=retail_object_detection_dino_meta.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=retail_object_detection_dino_meta.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=retail_object_detection_dino_meta.engine --batch=32

DINO COCO 模型名稱:dino_fan_small_ep12.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_dino_coco/version

指令語法:
* trtexec --onnx=dino_fan_small_ep12.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=dino_fan_small_ep12.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=dino_fan_small_ep12.engine --batch=32

GC-ViT ImageNet 模型名稱:gcvit_tiny_imagenet1k.pth
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_dino_imagenet/version

指令語法:
* trtexec --onnx=gc_vit_tiny.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=gc_vit_tiny.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=gc_vit_tiny.engine --batch=128

OCRNet 模型名稱:ocrnet_resnet50_pruned.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/ocrnet/version

指令語法:
* trtexec --onnx=ocrnet_resnet50_pruned.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=ocrnet_resnet50_pruned.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=ocrnet_resnet50_pruned.engine --batch=128

OCDNet 模型名稱:dcn_resnet18.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/ocdnet/version

指令語法:
* trtexec --onnx=dcn_resnet18.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=dcn_resnet18.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=dcn_resnet18.engine --batch=128

Optical Inspection 模型名稱:oi_model.onnx
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/optical_inspection/version

指令語法:
* trtexec --onnx=oi_model.onnx --verbose --useCudaGraph --useSpinWait --fp16 --saveEngine=oi_model.engine
* trtexec --verbose --useCudaGraph --useSpinWait --fp16 --loadEngine=oi_model.engine --batch=32