# Object Detection on Jetson Xavier NX
###### tags: `Jetson Xavier NX`

[TOC]

## Introduction
Nvidia provides the package of image recognition and object detection including ==Resnet, VGG, SSD etc.==
It's easy to start the object detection on Jetson single board and also create your own detection more rapidly.
It drastically reduce development time.

## Build the environment

Due to various mounts and devices needed use docker and run the container.

### Tutorial Video
{%youtube QXIwdsyK7Rw %}

### Procedure

* pull the jetson-inference on github.

$ git clone --recursive https://github.com/dusty-nv/jetson-inference

* change the directory.

$ cd jetson-inference

* Run the container.

$ docker/run.sh

:::info
==docker/run.sh== will automatically pull the correct container tag from DockerHub based on your currently-installed version of JetPack-L4T, and mount the appropriate data directories and devices so that you can use cameras/display/ect from within the container. It will also prompt you to download DNN models if you haven't already done so, which get mounted into the container to load. This initial setup is only done once.
:::

**Reference :**
https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-docker.md

## Test the Camera

### V4L2 cameras

$ video-viewer /dev/video0

### MIPI CSI cameras

$ video-viewer csi://0

video-viewer --input-width=1920 --input-height=1080 csi://0

**Reference:**
https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

## Run the detection

The command to run the object detection to detect images or videos.
The executive files provide C++ and python.

:::warning
:warning: The following commands are all run in the container.
:::

### Tutorial video
{%youtube obt60r8ZeB0 %}

### Images

cd build/aarch64/bin

./detectnet "images/cat_*.jpg" images/test/cat_%i.jpg

### Video

Using docker ==--volume==

docker/run.sh --volume $resource_file_path:$target_file_path

:::info
--volume : The file will exist in the container until it exit.
:::

detectnet --threshold=0.35 $video_file_path

>The default threshold is 0.5.

### Webcam

detectnet.py /dev/video0

* Adjust camera resolution.

detectnet.py --input-width=1920 --input-height=1080 /dev/video0

## Training Own Dataset

### Tutorial Video
{%youtube 2XMkPW_sIGg %}

### Collect Data

camera-capture /dev/video0

:::info
⚠️ The dataset annotation format : VOC(XML)
:::

### Training Data

$ cd jetson-inference/python/training/detection/ssd

$ python3 train_ssd.py --dataset-type=voc --data=data/<YOUR-DATASET> --model-dir=models/<MODEL-FILENAME>

| Argument | Default | Description |
| ------------ | ------- | ---------------------------------------------------------- |
| --data | data/ | The location of dataset. |
| --model-dir | models/ | directory to output the trained model checkpoints |
| --resume | None | path to an existing checkpoint to resume training from |
| --batch-size | 4 | try increasing depending on available memory |
| --epochs | 30 | up to 100 is desirable, but will increase training time |
| --workers | 2 | number of data loader threads (0 = disable multithreading) |

$ python3 onnx_export.py --model-dir=models/<YOUR-MODEL>

### Detect by own dataset

detectnet --model=models/$model_filename/ssd-mobilenet.onnx --labels=$NET/labels.txt \
          --input-blob=input_0 --output-cvg=scores --output-bbox=boxes \
            /dev/video0

## Build own detection code

import jetson.inference
import jetson.utils

net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)
camera = jetson.utils.videoSource("csi://0")      # '/dev/video0' for V4L2
display = jetson.utils.videoOutput("display://0") # 'my_video.mp4' for file

while display.IsStreaming():
	img = camera.Capture()
	detections = net.Detect(img)
	display.Render(img)
	display.SetStatus("Object Detection | Network {:.0f} FPS".format(net.GetNetworkFPS()))

### Save Detection Output Video

Add ==file://$filename.mp4== for saving detection output video.

savefile = jetson.utils.videoOutput("file://my_video.mp4") # 'file://my_video.mp4' for file

import jetson.inference
import jetson.utils

net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)
camera = jetson.utils.videoSource("/dev/video0")# '/dev/video0' for V4L2
display = jetson.utils.videoOutput("display://0")
savefile = jetson.utils.videoOutput("file://my_video.mp4") # 'file://my_video.mp4' for file

while display.IsStreaming():
        img = camera.Capture()
        detections = net.Detect(img)
        display.Render(img)
        display.SetStatus("Object Detection | Network {:.0f} FPS".format(net.GetNetworkFPS()))
        savefile.Render(img)
        savefile.SetStatus("Object Detection | Network {:.0f} FPS".format(net.GetNetworkFPS()))
        #print(img.width)
        #print(img.height)
        if len(detections)!=0:
            # print(detections[0].ClassID)
            # print(detections[0].Center[0])
            if detections[0].ClassID == 1:
                print("Person Bounding Box Area {}".format(detections[0].Area))

### Detection Parameters

print(detections[0])

**Output :**

Area: Area of bounding box
Bottom: Bottom bounding box coordinate`
Center: Center (x,y) coordinate of bounding box
ClassID:Class index of the detected object
Confidence: Confidence value of the detected object
Height: Height of bounding box
Instance: Instance index of the detected object
Left: Left bounding box coordinate
Right: Right bounding box coordinate
Top: Top bounding box coordinate
Width: Width of bounding box

For example to get the bounding box's center.

print(detections[0].Center)

### Image Parameters
當您在Python中分配圖像，或捕獲視頻源中的圖像時videoSource.Capture()，它將返回一個自包含的內存膠囊對象（類型為<jetson.utils.cudaImage>），無需複制底層內存即可傳遞該對象。
該cudaImage對象具有以下成員：

<jetson.utils.cudaImage>
  .ptr      # memory address (not typically used)
  .size     # size in bytes
  .shape    # (height,width,channels) tuple
  .width    # width in pixels
  .height   # height in pixels
  .channels # number of color channels
  .format   # format string
  .mapped   # true if ZeroCopy

For example to get the width and height of image.

print("Image's width : {}".format(img.width))
print("Image's height : {}".format(img.height))

## Reference

* https://github.com/dusty-nv/jetson-inference
* https://rawgit.com/dusty-nv/jetson-inference/dev/docs/html/python/jetson.inference.html#detectNet
* https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md#image-capsules-in-python