Merge pull request #14 from NVIDIA-ISAAC-ROS/release-dp-2

Isaac ROS 0.20.0 (DP2)
NVIDIA-ISAAC-ROS · Oct 19, 2022 · 2d73f40 · 2d73f40
2 parents b0b32c0 + 28fa663
commit 2d73f40
Show file tree

Hide file tree

Showing 18 changed files with 569 additions and 177 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,14 @@
+# Isaac ROS Contribution Rules
+
+Any contribution that you make to this repository will
+be under the Apache 2 License, as dictated by that
+[license](http://www.apache.org/licenses/LICENSE-2.0.html):
+
+> **5. Submission of Contributions.** Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
+
+Contributors must sign-off each commit by adding a `Signed-off-by: ...`
+line to commit messages to certify that they have the right to submit
+the code they are contributing to the project according to the
+[Developer Certificate of Origin (DCO)](https://developercertificate.org/).
+
+[//]: # (202201002)
diff --git a/LICENSE b/LICENSE
diff --git a/README.md b/README.md
@@ -3,20 +3,25 @@
 <div align="center"><img alt="Isaac ROS DetectNet Sample Output" src="resources/header-image.png" width="600px"/></div>
 
 ## Overview
+
 This repository provides a GPU-accelerated package for object detection based on [DetectNet](https://developer.nvidia.com/blog/detectnet-deep-neural-network-object-detection-digits/). Using a trained deep-learning model and a monocular camera, the `isaac_ros_detectnet` package can detect objects of interest in an image and provide bounding boxes. DetectNet is similar to other popular object detection models such as YOLOV3, FasterRCNN, SSD, and others while being efficient with multiple object classes in large images.
 
 ### Isaac ROS NITROS Acceleration
+
 This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
 
 ### Performance
-The performance results of benchmarking the prepared pipelines in this package on supported platforms are below:
 
+The performance results of benchmarking the prepared pipelines in this package on supported platforms are below:
 
-| Pipeline                   | AGX Orin  | AGX Xavier | x86_64 w/ RTX 3060 Ti |
-| -------------------------- | --------- | ---------- | --------------------- |
-| Isaac ROS Detectnet (544p) | 104 fps   | 79 fps     | 104 fps               |
+| Pipeline                   | AGX Orin           | Orin Nano        | x86_64 w/ RTX 3060 Ti |
+| -------------------------- | ------------------ | ---------------- | --------------------- |
+| Isaac ROS Detectnet (544p) | 225 fps <br> 7.7ms | 72 fps <br> 18ms | 450 fps <br> 3.2ms    |
 
 > **Note:** These numbers are reported with defaults parameter values found in [params.yaml](./isaac_ros_detectnet/config/params.yaml).
+
+These data have been collected per the methodology described [here](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/performance-summary.md#methodology).
+
 ### ROS2 Graph Configuration
 
 To run the DetectNet object detection inference, the following ROS2 nodes should be set up and running:
@@ -25,13 +30,14 @@ To run the DetectNet object detection inference, the following ROS2 nodes should
 
 1. **Isaac ROS DNN Image encoder**: This will take an image message and convert it to a tensor ([`TensorList`](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) that can be
    processed by the network.
-2. **Isaac ROS DNN Inference - Triton**: This will execute the DetectNet network and take as input the tensor from the DNN Image Encoder. 
+2. **Isaac ROS DNN Inference - Triton**: This will execute the DetectNet network and take as input the tensor from the DNN Image Encoder.
     > **Note:** The [Isaac ROS TensorRT](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/tree/main/isaac_ros_tensor_rt) package is not able to perform inference with DetectNet models at this time.
   
    The output will be a TensorList message containing the encoded detections. Use the parameters `model_name` and `model_repository_paths` to point to the model folder and set the model name. The `.plan` file should be located at `$model_repository_path/$model_name/1/model.plan`
 3. **Isaac ROS Detectnet Decoder**: This node will take the TensorList with encoded detections as input, and output `Detection2DArray` messages for each frame. See the following section for the parameters.
 
 ## Table of Contents
+
 - [Isaac ROS Object Detection](#isaac-ros-object-detection)
   - [Overview](#overview)
     - [Isaac ROS NITROS Acceleration](#isaac-ros-nitros-acceleration)
@@ -44,7 +50,6 @@ To run the DetectNet object detection inference, the following ROS2 nodes should
   - [Quickstart](#quickstart)
   - [Next Steps](#next-steps)
     - [Try More Examples](#try-more-examples)
-    - [Use Different Models](#use-different-models)
     - [Customize your Dev Environment](#customize-your-dev-environment)
   - [Package Reference](#package-reference)
     - [`isaac_ros_detectnet`](#isaac_ros_detectnet)
@@ -58,26 +63,28 @@ To run the DetectNet object detection inference, the following ROS2 nodes should
   - [Updates](#updates)
 
 ## Latest Update
-Update 2022-08-31: Update to use [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/) and to be compatible with JetPack 5.0.2
+
+Update 2022-10-19: Updated OSS licensing
 
 ## Supported Platforms
+
 This package is designed and tested to be compatible with ROS2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) or an x86_64 system with an NVIDIA GPU.
 
 > **Note**: Versions of ROS2 earlier than Humble are **not** supported. This package depends on specific ROS2 implementation features that were only introduced beginning with the Humble release.
 
-
-| Platform | Hardware                                                                                                                                                                                                | Software                                                                                                             | Notes                                                                                                                                                                                   |
-| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Jetson   | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/)<br/>[Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.0.2](https://developer.nvidia.com/embedded/jetpack)                                                       | For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
-| x86_64   | NVIDIA GPU                                                                                                                                                                                              | [Ubuntu 20.04+](https://releases.ubuntu.com/20.04/) <br> [CUDA 11.6.1+](https://developer.nvidia.com/cuda-downloads) |
-
+| Platform | Hardware                                                                                                                                                                                                 | Software                                                                                                             | Notes                                                                                                                                                                                   |
+| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Jetson   | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) <br> [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.0.2](https://developer.nvidia.com/embedded/jetpack)                                                       | For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
+| x86_64   | NVIDIA GPU                                                                                                                                                                                               | [Ubuntu 20.04+](https://releases.ubuntu.com/20.04/) <br> [CUDA 11.6.1+](https://developer.nvidia.com/cuda-downloads) |
 
 ### Docker
+
 To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following [these steps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md). This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
 
 > **Note:** All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
 
 ## Quickstart
+
 1. Set up your development environment by following the instructions [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md).
 2. Clone this repository and its dependencies under `~/workspaces/isaac_ros-dev/src`.
 
@@ -102,47 +109,59 @@ To simplify development, we strongly recommend leveraging the Isaac ROS Dev Dock
     ```
 
 3. Pull down a ROS Bag of sample data:
+
     ```bash
     cd ~/workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
       git lfs pull -X "" -I "resources/rosbags"
     ```
+
 4. Launch the Docker container using the `run_dev.sh` script:
+
     ```bash
     cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
       ./scripts/run_dev.sh
     ```
+
 5. Inside the container, build and source the workspace:
+
     ```bash
     cd /workspaces/isaac_ros-dev && \
       colcon build --symlink-install && \
       source install/setup.bash
     ```
+
 6. (Optional) Run tests to verify complete and correct installation:
+
     ```bash
     colcon test --executor sequential
     ```
+
 7. Run the quickstart setup script which will download the [PeopleNet Model](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet) from NVIDIA GPU Cloud(NGC)
+
     ```bash
     cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
       ./scripts/setup_model.sh --height 632 --width 1200 --config-file resources/quickstart_config.pbtxt
     ```
 
 8. Run the following launch file to spin up a demo of this package:
+
     ```bash
     cd /workspaces/isaac_ros-dev && \
       ros2 launch isaac_ros_detectnet isaac_ros_detectnet_quickstart.launch.py
     ```
-9.  Visualize and validate the output of the package in the `rqt_image_view` window. After about a minute, your output should look like this:
+
+9. Visualize and validate the output of the package in the `rqt_image_view` window. After about a minute, your output should look like this:
 
     ![DetectNet output image showing a tennis ball correctly identified](resources/rqt_visualizer.png "RQT showing detection boxes of an NVIDIA Mug and a tennis ball from simulation using DetectNet")
 
 ## Next Steps
+
 ### Try More Examples
+
 To continue your exploration, check out the following suggested examples:
-- [Tutorial with Isaac Sim](docs/tutorial-isaac-sim.md)
 
-### Use Different Models
-Click [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/docs/model-preparation.md) for more information about how to use NGC models.
+- [Tutorial with Isaac Sim](docs/tutorial-isaac-sim.md)
+- [Tutorial with Custom Model](docs/tutorial-custom-model.md) For more info click [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/docs/model-preparation.md)
 
 This package only supports models based on the `Detectnet_v2` architecture. Some of the [supported DetectNet models](https://catalog.ngc.nvidia.com/?filters=&orderBy=scoreDESC&query=DetectNet) from NGC:
 
@@ -153,12 +172,14 @@ This package only supports models based on the `Detectnet_v2` architecture. Some
 | [DashCamNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:dashcamnet)       | Identify objects from a moving object                  |
 | [FaceDetectIR](https://ngc.nvidia.com/catalog/models/nvidia:tao:facedetectir)   | Detect faces in a dark environment with IR camera      |
 
-
 ### Customize your Dev Environment
+
 To customize your development environment, reference [this guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/modify-dockerfile.md).
 
 ## Package Reference
+
 ### `isaac_ros_detectnet`
+
 #### Usage
 
 ```bash
@@ -182,26 +203,32 @@ ros2 launch isaac_ros_detectnet isaac_ros_detectnet.launch.py label_list:=<list
 | `bounding_box_offset`         | `double`   | `0.0`                       | Bounding box offset for both X and Y dimensions.                                                                                                                                                                        |
 
 #### ROS Topics Subscribed
+
 | ROS Topic    | Interface                                                                                                                                                         | Description                                                     |
 | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
 | `tensor_sub` | [isaac_ros_tensor_list_interfaces/TensorList](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) | The tensor that represents the inferred aligned bounding boxes. |
 
 #### ROS Topics Published
+
 | ROS Topic              | Interface                                                                                                        | Description                                        |
 | ---------------------- | ---------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
 | `detectnet/detections` | [vision_msgs/Detection2DArray](https://github.com/ros-perception/vision_msgs/blob/ros2/msg/Detection2DArray.msg) | Aligned image bounding boxes with detection class. |
 
-
 ## Troubleshooting
+
 ### Isaac ROS Troubleshooting
+
 For solutions to problems with Isaac ROS, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/troubleshooting.md).
 
 ### Deep Learning Troubleshooting
+
 For solutions to problems with using DNN models, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/docs/troubleshooting.md).
 
 ## Updates
+
 | Date       | Changes                                                                               |
 | ---------- | ------------------------------------------------------------------------------------- |
+| 2022-10-19 | Updated OSS licensing                                                                 |
 | 2022-08-31 | Update to use NITROS for improved performance and to be compatible with JetPack 5.0.2 |
 | 2022-06-30 | Support for ROS2 Humble and miscellaneous bug fixes                                   |
 | 2022-03-21 | Initial release                                                                       |
diff --git a/docs/tutorial-custom-model.md b/docs/tutorial-custom-model.md
@@ -0,0 +1,39 @@
+# Tutorial for DetectNet with a Custom Model
+
+## Overview
+
+This tutorial walks you through how to use a different [DetectNet Model](https://catalog.ngc.nvidia.com/models?filters=&orderBy=dateModifiedDESC&query=detectnet) with [isaac_ros_detectnet](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) for object detection.
+
+## Tutorial Walkthrough
+
+1. Complete the [Quickstart section](../README.md#quickstart) in the main README.
+2. Choose one of the detectnet model that is [listed here](https://catalog.ngc.nvidia.com/models?filters=&orderBy=dateModifiedDESC&query=detectnet&page=0&pageSize=25)
+3. Create a config file. Use `resources/quickstart_config.pbtxt` as a template. The datatype can be found in the overview tab of the model page. The `input/dims` should be the size of the raw input images. It can be different for the same model. The `output/dims` dimensions can be calculated as `round(input_dims/max_batch_size)`. Place this config file in the `isaac_ros_detectnet/resources` directory. You can find more information about the config file [here](https://github.com/NVIDIA-AI-IOT/tao-toolkit-triton-apps/blob/main/docs/configuring_the_client.md#configuring-the-detectnet_v2-model-entry-in-the-model-repository)
+4. Run the following command with the required input parameters:
+
+    ```bash
+    cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
+      ./scripts/setup_model.sh --height 720 --width 1280 --config-file resources/isaac_sim_config.pbtxt
+    ```
+
+    Parameters:
+
+    `--model-link` : Get the wget link to the specific model version under the file browser tab in the page. Click on the download button on the top right and select WGET. This will copy the commend to you clipboard. Paste this in a text editor and extract only the hyperlink. eg: `https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplenet/versions/deployable_quantized_v2.5/zip`
+
+    `--model-file-name` : The name of the .etl file found in the file browser tab of the model page. eg: `resnet34_peoplenet_int8.etlt`
+
+    `--height` : height dimension of the input image eg: `632`
+
+    `--width` : width dimension of the input image. eg: `1200`
+
+    `--config-file` : relative path to the config file mentioned in step 3. eg: `isaac_ros_detectnet/resources/peoplenet_config.pbtxt`
+    --precision : type/precision of model found in the overview tag of the model page. eg: `int8`
+
+    `--output-layers`: output layers seperated by commas that can be found from the txt file in the file browser tab of the model page. eg: `output_cov/Sigmoid,output_bbox/BiasAdd`
+5. Replace lines 32 and 33 in [isaac_ros_detectnet.launch.py](../isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py#L32-33) with the input image dimensions
+6. Run the following command:
+
+    ```bash
+    cd /workspaces/isaac_ros-dev && \
+      ros2 launch isaac_ros_detectnet isaac_ros_detectnet.launch.py
+    ```