+Hardware-accelerated, deep learned model support for object detection including DetectNet.
-## Overview
-
-Isaac ROS Object Detection contains an ROS 2 package to perform object detection. `isaac_ros_detectnet` provides a method for spatial classification using bounding boxes with an input image. Classification is performed by a GPU-accelerated [DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2) model. The output prediction can be used by perception functions to understand the presence and spatial location of an object in an image.
-
-
-
-`isaac_ros_detectnet` is used in a graph of nodes to provide a bounding box detection array with object classes from an input image. A [DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2) model is required to produce the detection array. Input images may need to be cropped and resized to maintain the aspect ratio and match the input resolution of DetectNet; image resolution may be reduced to improve DNN inference performance, which typically scales directly with the number of pixels in the image. `isaac_ros_dnn_image_encoder` provides a DNN encoder to process the input image into Tensors for the DetectNet model. Prediction results are clustered in the DNN decoder to group multiple detections on the same object. Output is provided as a detection array with object classes.
-
-DNNs have a minimum number of pixels that need to be visible on the object to provide a classification prediction. If a person cannot see the object in the image, it’s unlikely the DNN will. Reducing input resolution to reduce compute may reduce what is detected in the image. For example, a 1920x1080 image containing a distant person occupying 1k pixels (64x16) would have 0.25K pixels (32x8) when downscaled by 1/2 in both X and Y. The DNN may detect the person with the original input image, which provides 1K pixels for the person, and fail to detect the same person in the downscaled resolution, which only provides 0.25K pixels for the person.
-
-> **Note**: DetectNet is similar to other popular object detection models such as YOLOV3, FasterRCNN, and SSD, while being efficient at detecting multiple object classes in large images.
-
-
-
-Object detection classifies a rectangle of pixels as containing an object, whereas image segmentation provides more information and uses more compute to produce a classification per pixel. Object detection is used to know if, and where in a 2D image, the object exists. If a 3D spacial understanding or size of an object in pixels is required, use image segmentation.
-
-### DNN Models
+
+
-To perform DNN inferencing a DNN model is required. NGC provides [DetectNet pre-trained models](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2) for use in your robotics application. Using [TAO](https://developer.nvidia.com/tao-toolkit) these pre-trained models can be fine-tuned for your application.
+## Overview
-### Isaac ROS NITROS Acceleration
+[Isaac ROS Object Detection](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) contains an ROS 2 package to perform object
+detection. `isaac_ros_detectnet` provides a method for spatial
+classification using bounding boxes with an input image. Classification
+is performed by a GPU-accelerated
+[DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2)
+model. The output prediction can be used by perception functions to
+understand the presence and spatial location of an object in an image.
+
+
+
+`isaac_ros_detectnet` is used in a graph of nodes to provide a
+bounding box detection array with object classes from an input image. A
+[DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2)
+model is required to produce the detection array. Input images may need
+to be cropped and resized to maintain the aspect ratio and match the
+input resolution of DetectNet; image resolution may be reduced to
+improve DNN inference performance, which typically scales directly with
+the number of pixels in the image. `isaac_ros_dnn_image_encoder`
+provides a DNN encoder to process the input image into Tensors for the
+DetectNet model. Prediction results are clustered in the DNN decoder to
+group multiple detections on the same object. Output is provided as a
+detection array with object classes.
+
+DNNs have a minimum number of pixels that need to be visible on the
+object to provide a classification prediction. If a person cannot see
+the object in the image, it’s unlikely the DNN will. Reducing input
+resolution to reduce compute may reduce what is detected in the image.
+For example, a 1920x1080 image containing a distant person occupying 1k
+pixels (64x16) would have 0.25K pixels (32x8) when downscaled by 1/2 in
+both X and Y. The DNN may detect the person with the original input
+image, which provides 1K pixels for the person, and fail to detect the
+same person in the downscaled resolution, which only provides 0.25K
+pixels for the person.
+
+> [!Note]
+> DetectNet is similar to other popular object detection
+> models such as YOLOV3, FasterRCNN, and SSD, while being efficient at
+> detecting multiple object classes in large images.
+
+
+Object detection classifies a rectangle of pixels as containing an
+object, whereas image segmentation provides more information and uses
+more compute to produce a classification per pixel. Object detection is
+used to know if, and where in a 2D image, the object exists. If a 3D
+spacial understanding or size of an object in pixels is required, use
+image segmentation.
+
+## Isaac ROS NITROS Acceleration
This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
-### Performance
-
-The performance results of benchmarking the prepared pipelines in this package on supported platforms are below:
-
-| Sample Graph | Input Size | AGX Orin | Orin NX | Orin Nano 8GB | x86_64 w/ RTX 4060 Ti |
-| --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
-| [DetectNet Object Detection Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts//isaac_ros_detectnet_graph.py) | 544p | [252 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_detectnet_graph-agx_orin.json) 8.7 ms | [110 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_detectnet_graph-orin_nx.json) 13 ms | [77.9 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_detectnet_graph-orin_nano_emul.json) 18 ms | -- |
-
-
-> **Note**: These numbers are reported with defaults parameter values found in [params.yaml](./isaac_ros_detectnet/config/params.yaml).
-
-These data have been collected per the methodology described [here](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/performance-summary.md#methodology).
-
-### ROS 2 Graph Configuration
-
-To run the DetectNet object detection inference, the following ROS 2 nodes should be set up and running:
-
-
-
-1. **Isaac ROS DNN Image encoder**: This will take an image message and convert it to a tensor ([`TensorList`](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) that can be
- processed by the network.
-2. **Isaac ROS DNN Inference - Triton**: This will execute the DetectNet network and take as input the tensor from the DNN Image Encoder.
- > **Note**: The [Isaac ROS TensorRT](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/tree/main/isaac_ros_tensor_rt) package is not able to perform inference with DetectNet models at this time.
-
- The output will be a TensorList message containing the encoded detections. Use the parameters `model_name` and `model_repository_paths` to point to the model folder and set the model name. The `.plan` file should be located at `$model_repository_path/$model_name/1/model.plan`
-3. **Isaac ROS Detectnet Decoder**: This node will take the TensorList with encoded detections as input, and output `Detection2DArray` messages for each frame. See the following section for the parameters.
-
-## Table of Contents
-
-- [Isaac ROS Object Detection](#isaac-ros-object-detection)
- - [Overview](#overview)
- - [DNN Models](#dnn-models)
- - [Isaac ROS NITROS Acceleration](#isaac-ros-nitros-acceleration)
- - [Performance](#performance)
- - [ROS 2 Graph Configuration](#ros-2-graph-configuration)
- - [Table of Contents](#table-of-contents)
- - [Latest Update](#latest-update)
- - [Supported Platforms](#supported-platforms)
- - [Docker](#docker)
- - [Quickstart](#quickstart)
- - [Next Steps](#next-steps)
- - [Try More Examples](#try-more-examples)
- - [Customize your Dev Environment](#customize-your-dev-environment)
- - [Package Reference](#package-reference)
- - [`isaac_ros_detectnet`](#isaac_ros_detectnet)
- - [Usage](#usage)
- - [ROS Parameters](#ros-parameters)
- - [ROS Topics Subscribed](#ros-topics-subscribed)
- - [ROS Topics Published](#ros-topics-published)
- - [Troubleshooting](#troubleshooting)
- - [Isaac ROS Troubleshooting](#isaac-ros-troubleshooting)
- - [Deep Learning Troubleshooting](#deep-learning-troubleshooting)
- - [Updates](#updates)
-
-## Latest Update
-
-Update 2023-05-25: Performance improvements.
-
-## Supported Platforms
-
-This package is designed and tested to be compatible with ROS 2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) or an x86_64 system with an NVIDIA GPU.
-
-> **Note**: Versions of ROS 2 earlier than Humble are **not** supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
-
-| Platform | Hardware | Software | Notes |
-| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Jetson | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.1.1](https://developer.nvidia.com/embedded/jetpack) | For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
-| x86_64 | NVIDIA GPU | [Ubuntu 20.04+](https://releases.ubuntu.com/20.04/) [CUDA 11.8](https://developer.nvidia.com/cuda-downloads) |
-
-### Docker
-
-To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following [these steps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md). This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
-
-> **Note**: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
-
-## Quickstart
-
-1. Set up your development environment by following the instructions [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md).
-2. Clone this repository and its dependencies under `~/workspaces/isaac_ros-dev/src`.
-
- ```bash
- cd ~/workspaces/isaac_ros-dev/src
- ```
-
- ```bash
- git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection
- ```
-
- ```bash
- git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference
- ```
-
- ```bash
- git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_image_pipeline
- ```
-
- ```bash
- git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros
- ```
-
- ```bash
- git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common
- ```
-
-3. Pull down a ROS Bag of sample data:
-
- ```bash
- cd ~/workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
- git lfs pull -X "" -I "resources/rosbags"
- ```
-
-4. Launch the Docker container using the `run_dev.sh` script:
-
- ```bash
- cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
- ./scripts/run_dev.sh
- ```
-
-5. Inside the container, build and source the workspace:
-
- ```bash
- cd /workspaces/isaac_ros-dev && \
- colcon build --symlink-install && \
- source install/setup.bash
- ```
-
-6. (Optional) Run tests to verify complete and correct installation:
-
- ```bash
- colcon test --executor sequential
- ```
-
-7. Run the quickstart setup script which will download the [PeopleNet Model](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet) from NVIDIA GPU Cloud(NGC)
-
- ```bash
- cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
- ./scripts/setup_model.sh --height 632 --width 1200 --config-file resources/quickstart_config.pbtxt
- ```
-
-8. Run the following launch file to spin up a demo of this package:
-
- ```bash
- cd /workspaces/isaac_ros-dev && \
- ros2 launch isaac_ros_detectnet isaac_ros_detectnet_quickstart.launch.py
- ```
-
-9. Visualize and validate the output of the package in the `rqt_image_view` window. After about a minute, your output should look like this:
-
- 
-
-## Next Steps
-
-### Try More Examples
-
-To continue your exploration, check out the following suggested examples:
-
-- [Tutorial with Isaac Sim](docs/tutorial-isaac-sim.md)
-- [Tutorial with Custom Model](docs/tutorial-custom-model.md) For more info click [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/docs/model-preparation.md)
-
-This package only supports models based on the `Detectnet_v2` architecture. Some of the [supported DetectNet models](https://catalog.ngc.nvidia.com/?filters=&orderBy=scoreDESC&query=DetectNet) from NGC:
-
-| Model Name | Use Case |
-| ------------------------------------------------------------------------------- | ------------------------------------------------------ |
-| [TrafficCamNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:trafficcamnet) | Detect and track cars |
-| [PeopleNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:peoplenet) | People counting, heatmap generation, social distancing |
-| [DashCamNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:dashcamnet) | Identify objects from a moving object |
-| [FaceDetectIR](https://ngc.nvidia.com/catalog/models/nvidia:tao:facedetectir) | Detect faces in a dark environment with IR camera |
-
-### Customize your Dev Environment
-
-To customize your development environment, reference [this guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/modify-dockerfile.md).
-
-## Package Reference
-
-### `isaac_ros_detectnet`
-
-#### Usage
-
-```bash
-ros2 launch isaac_ros_detectnet isaac_ros_detectnet.launch.py label_list:= enable_confidence_threshold:= enable_bbox_area_threshold:= enable_dbscan_clustering:= confidence_threshold:= min_bbox_area:= dbscan_confidence_threshold:= dbscan_eps:= dbscan_min_boxes:= dbscan_enable_athr_filter:= dbscan_threshold_athr:= dbscan_clustering_algorithm:= bounding_box_scale:= bounding_box_offset:=
-```
-
-#### ROS Parameters
-
-| ROS Parameter | Type | Default | Description |
-| ----------------------------- | ---------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `label_list` | `string[]` | `{"person", "bag", "face"}` | The list of labels. These are loaded from labels.txt(downloaded with the model) |
-| `confidence_threshold` | `double` | `0.35` | The min value of confidence used to threshold detections before clustering |
-| `min_bbox_area` | `double` | `100` | The min value of bouding box area used to threshold detections before clustering |
-| `dbscan_confidence_threshold` | `double` | `0.35` | Holds the epsilon to control merging of overlapping boxes. Refer to OpenCV groupRectangles and DBSCAN documentation for more information on epsilon. |
-| `dbscan_eps` | `double` | `0.7` | Holds the epsilon to control merging of overlapping boxes. Refer to OpenCV groupRectangles and DBSCAN documentation for more information on epsilon. |
-| `dbscan_min_boxes` | `int` | `1` | The minimum number of boxes to return. |
-| `dbscan_enable_athr_filter` | `int` | `0` | Enables the area-to-hit ratio (ATHR) filter. The ATHR is calculated as: **ATHR = sqrt(clusterArea) / nObjectsInCluster.** |
-| `dbscan_threshold_athr` | `double` | `0.0` | The `area-to-hit` ratio threshold. |
-| `dbscan_clustering_algorithm` | `int` | `1` | The clustering algorithm selection. (`1`: Enables DBScan clustering, `2`: Enables Hybrid clustering, resulting in more boxes that will need to be processed with NMS or other means of reducing overlapping detections. |
-| `bounding_box_scale` | `double` | `35.0` | The scale parameter, which should match the training configuration. |
-| `bounding_box_offset` | `double` | `0.0` | Bounding box offset for both X and Y dimensions. |
-
-#### ROS Topics Subscribed
-
-| ROS Topic | Interface | Description |
-| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
-| `tensor_sub` | [isaac_ros_tensor_list_interfaces/TensorList](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) | The tensor that represents the inferred aligned bounding boxes. |
+## Performance
-#### ROS Topics Published
+| Sample Graph
|
-| ROS Topic | Interface | Description |
-| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
-| `detectnet/detections` | [vision_msgs/Detection2DArray](https://github.com/ros-perception/vision_msgs/blob/ros2/vision_msgs/msg/Detection2DArray.msg) | Aligned image bounding boxes with detection class. |
+---
-## Troubleshooting
+## Documentation
-### Isaac ROS Troubleshooting
+Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/index.html) to learn how to use this repository.
-For solutions to problems with Isaac ROS, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/troubleshooting.md).
+---
-### Deep Learning Troubleshooting
+## Packages
-For solutions to problems with using DNN models, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/docs/troubleshooting.md).
+* [`isaac_ros_detectnet`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html)
+ * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#quickstart)
+ * [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#try-more-examples)
+ * [ROS 2 Graph Configuration](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#ros-2-graph-configuration)
+ * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#troubleshooting)
+ * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_detectnet/index.html#api)
+* [`isaac_ros_yolov8`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html)
+ * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#quickstart)
+ * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_object_detection/isaac_ros_yolov8/index.html#api)
-## Updates
+## Latest
-| Date | Changes |
-| ---------- | ------------------------------------------------------------------------------------- |
-| 2023-05-25 | Performance improvements |
-| 2023-04-05 | Source available GXF extensions |
-| 2022-10-19 | Updated OSS licensing |
-| 2022-08-31 | Update to use NITROS for improved performance and to be compatible with JetPack 5.0.2 |
-| 2022-06-30 | Support for ROS 2 Humble and miscellaneous bug fixes |
-| 2022-03-21 | Initial release |
+Update 2023-10-18: Adding NITROS YOLOv8 decoder.
diff --git a/docs/tutorial-custom-model.md b/docs/tutorial-custom-model.md
deleted file mode 100644
index a729834..0000000
--- a/docs/tutorial-custom-model.md
+++ /dev/null
@@ -1,39 +0,0 @@
-# Tutorial for DetectNet with a Custom Model
-
-## Overview
-
-This tutorial walks you through how to use a different [DetectNet Model](https://catalog.ngc.nvidia.com/models?filters=&orderBy=dateModifiedDESC&query=detectnet) with [isaac_ros_detectnet](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) for object detection.
-
-## Tutorial Walkthrough
-
-1. Complete the [Quickstart section](../README.md#quickstart) in the main README.
-2. Choose one of the detectnet model that is [listed here](https://catalog.ngc.nvidia.com/models?filters=&orderBy=dateModifiedDESC&query=detectnet&page=0&pageSize=25)
-3. Create a config file. Use `resources/quickstart_config.pbtxt` as a template. The datatype can be found in the overview tab of the model page. The `input/dims` should be the size of the raw input images. It can be different for the same model. The `output/dims` dimensions can be calculated as `round(input_dims/max_batch_size)`. Place this config file in the `isaac_ros_detectnet/resources` directory. You can find more information about the config file [here](https://github.com/NVIDIA-AI-IOT/tao-toolkit-triton-apps/blob/main/docs/configuring_the_client.md#configuring-the-detectnet_v2-model-entry-in-the-model-repository)
-4. Run the following command with the required input parameters:
-
- ```bash
- cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
- ./scripts/setup_model.sh --height 720 --width 1280 --config-file resources/isaac_sim_config.pbtxt
- ```
-
- Parameters:
-
- `--model-link` : Get the wget link to the specific model version under the file browser tab in the page. Click on the download button on the top right and select WGET. This will copy the commend to you clipboard. Paste this in a text editor and extract only the hyperlink. eg: `https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplenet/versions/deployable_quantized_v2.5/zip`
-
- `--model-file-name` : The name of the .etl file found in the file browser tab of the model page. eg: `resnet34_peoplenet_int8.etlt`
-
- `--height` : height dimension of the input image eg: `632`
-
- `--width` : width dimension of the input image. eg: `1200`
-
- `--config-file` : relative path to the config file mentioned in step 3. eg: `isaac_ros_detectnet/resources/peoplenet_config.pbtxt`
- --precision : type/precision of model found in the overview tag of the model page. eg: `int8`
-
- `--output-layers`: output layers seperated by commas that can be found from the txt file in the file browser tab of the model page. eg: `output_cov/Sigmoid,output_bbox/BiasAdd`
-5. Replace lines 32 and 33 in [isaac_ros_detectnet.launch.py](../isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py#L32-33) with the input image dimensions
-6. Run the following command:
-
- ```bash
- cd /workspaces/isaac_ros-dev && \
- ros2 launch isaac_ros_detectnet isaac_ros_detectnet.launch.py
- ```
diff --git a/docs/tutorial-isaac-sim.md b/docs/tutorial-isaac-sim.md
deleted file mode 100644
index 8c75a96..0000000
--- a/docs/tutorial-isaac-sim.md
+++ /dev/null
@@ -1,53 +0,0 @@
-# Tutorial for DNN Object Detection with Isaac Sim
-
-
-
-## Overview
-
-This tutorial walks you through a pipeline for object(people) detection using [DetectNet](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) consuming images from Isaac Sim.
-
-Last validated with [Isaac Sim 2022.2.1](https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/release_notes.html#id1)
-
-## Tutorial Walkthrough
-
-1. Complete the [Quickstart section](../README.md#quickstart) in the main README.
-2. Launch the Docker container using the `run_dev.sh` script:
-
- ```bash
- cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
- ./scripts/run_dev.sh
- ```
-
-3. Inside the container, build and source the workspace:
-
- ```bash
- cd /workspaces/isaac_ros-dev && \
- colcon build --symlink-install && \
- source install/setup.bash
- ```
-
-4. Run the setup script to download the [PeopleNet Model](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet) from NVIDIA GPU Cloud(NGC) and convert it to a .etlt file
-
- ```bash
- cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \
- ./scripts/setup_model.sh --height 720 --width 1280 --config-file resources/isaac_sim_config.pbtxt
- ```
-
-5. Launch the pre-composed pipeline launchfile:
-
- ```bash
- cd /workspaces/isaac_ros-dev && \
- ros2 launch isaac_ros_detectnet isaac_ros_detectnet_isaac_sim.launch.py
- ```
-
-6. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
-7. Open up the Isaac ROS Common USD scene (using the *Content* tab) located at:
- ```text
- http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2022.2.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd
- ```
- And wait for it to load completely.
-8. Go to the *Stage* tab and select `/World/Carter_ROS`, then in *Property* tab *-> Transform -> Translate* set *X* and *Y* both to `0.0`.
-
-9. Press **Play** to start publishing data from Isaac Sim.
-
-10. You should see the image from Isaac Sim with the rectangles overlayed over detected people in the frame.
diff --git a/isaac_ros_detectnet/CMakeLists.txt b/isaac_ros_detectnet/CMakeLists.txt
index bddbe8d..9710830 100644
--- a/isaac_ros_detectnet/CMakeLists.txt
+++ b/isaac_ros_detectnet/CMakeLists.txt
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2022-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@
#
# SPDX-License-Identifier: Apache-2.0
-cmake_minimum_required(VERSION 3.23.2)
+cmake_minimum_required(VERSION 3.22.1)
project(isaac_ros_detectnet LANGUAGES C CXX)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
@@ -43,6 +43,12 @@ if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
ament_lint_auto_find_test_dependencies()
+
+ # The FindPythonInterp and FindPythonLibs modules are removed
+ if(POLICY CMP0148)
+ cmake_policy(SET CMP0148 OLD)
+ endif()
+
find_package(launch_testing_ament_cmake REQUIRED)
add_launch_test(test/isaac_ros_detectnet_pol_test.py TIMEOUT "600")
endif()
diff --git a/isaac_ros_detectnet/gxf/detectnet/CMakeLists.txt b/isaac_ros_detectnet/gxf/detectnet/CMakeLists.txt
index cfc35d0..30e5df7 100644
--- a/isaac_ros_detectnet/gxf/detectnet/CMakeLists.txt
+++ b/isaac_ros_detectnet/gxf/detectnet/CMakeLists.txt
@@ -24,7 +24,7 @@ find_package(GXF ${ISAAC_ROS_GXF_VERSION} MODULE REQUIRED
std
)
find_package(isaac_ros_nitros_detection2_d_array_type REQUIRED)
-include(YamlCpp)
+find_package(yaml-cpp)
# DetectNet extension
add_library(gxf_detectnet SHARED
diff --git a/isaac_ros_detectnet/gxf/detectnet/deepstream_utils/nvdsinfer/include/nvdsinfer.h b/isaac_ros_detectnet/gxf/detectnet/deepstream_utils/nvdsinfer/include/nvdsinfer.h
index b06bd3d..c8cdd94 100644
--- a/isaac_ros_detectnet/gxf/detectnet/deepstream_utils/nvdsinfer/include/nvdsinfer.h
+++ b/isaac_ros_detectnet/gxf/detectnet/deepstream_utils/nvdsinfer/include/nvdsinfer.h
@@ -287,11 +287,14 @@ typedef enum
/**
* Holds full dimensions (including batch size) for a layer.
*/
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wmissing-field-initializers"
typedef struct
{
int batchSize = 0;
NvDsInferDims dims = {0};
} NvDsInferBatchDims;
+#pragma GCC diagnostic pop
/**
* Extended structure for bound layer information which additionally includes
diff --git a/isaac_ros_detectnet/gxf/detectnet/detectnet_decoder.cpp b/isaac_ros_detectnet/gxf/detectnet/detectnet_decoder.cpp
index 08645e8..e83668d 100644
--- a/isaac_ros_detectnet/gxf/detectnet/detectnet_decoder.cpp
+++ b/isaac_ros_detectnet/gxf/detectnet/detectnet_decoder.cpp
@@ -263,18 +263,18 @@ gxf_result_t DetectnetDecoder::tick() noexcept
// TODO(ashwinvk): Do not copy data to host and perform decoding using cuda
// copy memory to host
- float cov_tensor_arr[cov_tensor->size() / sizeof(float)]; // since data in tensor is kFloat32
+ std::unique_ptr cov_tensor_arr(new float[cov_tensor->element_count()]);
const cudaError_t cuda_error_cov_tensor = cudaMemcpy(
- &cov_tensor_arr, cov_tensor->pointer(),
+ cov_tensor_arr.get(), cov_tensor->pointer(),
cov_tensor->size(), cudaMemcpyDeviceToHost);
if (cuda_error_cov_tensor != cudaSuccess) {
GXF_LOG_ERROR("Error while copying kernel: %s", cudaGetErrorString(cuda_error_cov_tensor));
return GXF_FAILURE;
}
- float bbox_tensor_arr[bbox_tensor->size() / sizeof(float)]; // since data in tensor is kFloat32
+ std::unique_ptr bbox_tensor_arr(new float[bbox_tensor->element_count()]);
const cudaError_t cuda_error_bbox_tensor = cudaMemcpy(
- &bbox_tensor_arr, bbox_tensor->pointer(),
+ bbox_tensor_arr.get(), bbox_tensor->pointer(),
bbox_tensor->size(), cudaMemcpyDeviceToHost);
if (cuda_error_bbox_tensor != cudaSuccess) {
GXF_LOG_ERROR("Error while copying kernel: %s", cudaGetErrorString(cuda_error_bbox_tensor));
@@ -342,7 +342,7 @@ gxf_result_t DetectnetDecoder::tick() noexcept
// check if object_class is out of range for label_list_
if (static_cast(object_class) >= label_list_.get().size()) {
GXF_LOG_ERROR(
- "[DetectNet Decoder] object_class %i is out of range for provided label_list_ of size %i", object_class,
+ "[DetectNet Decoder] object_class %i is out of range for provided label_list_ of size %lu", object_class,
label_list_.get().size());
return GXF_FAILURE;
}
@@ -369,7 +369,7 @@ gxf_result_t DetectnetDecoder::tick() noexcept
} else {
GXF_LOG_ERROR(
"Invalid value for dbscan_clustering_algorithm: %i",
- dbscan_clustering_algorithm_);
+ dbscan_clustering_algorithm_.get());
return GXF_FAILURE;
}
NvDsInferDBScanDestroy(dbscan_hdl);
diff --git a/isaac_ros_detectnet/include/isaac_ros_detectnet/detectnet_decoder_node.hpp b/isaac_ros_detectnet/include/isaac_ros_detectnet/detectnet_decoder_node.hpp
index 3d6b047..2396c2e 100644
--- a/isaac_ros_detectnet/include/isaac_ros_detectnet/detectnet_decoder_node.hpp
+++ b/isaac_ros_detectnet/include/isaac_ros_detectnet/detectnet_decoder_node.hpp
@@ -1,5 +1,5 @@
// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
diff --git a/isaac_ros_detectnet/isaac_ros_detectnet/__init__.py b/isaac_ros_detectnet/isaac_ros_detectnet/__init__.py
index e69de29..6bbcb13 100644
--- a/isaac_ros_detectnet/isaac_ros_detectnet/__init__.py
+++ b/isaac_ros_detectnet/isaac_ros_detectnet/__init__.py
@@ -0,0 +1,16 @@
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
diff --git a/isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py b/isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py
index e8dad26..b55f6f4 100644
--- a/isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py
+++ b/isaac_ros_detectnet/launch/isaac_ros_detectnet.launch.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -35,11 +35,16 @@ def generate_launch_description():
encoder_node = ComposableNode(
name='dnn_image_encoder',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
parameters=[{
+ 'input_image_width': 1200,
+ 'input_image_height': 632,
'network_image_width': 1200,
- 'network_image_height': 632
+ 'network_image_height': 632,
+ 'image_mean': [0.0, 0.0, 0.0],
+ 'image_stddev': [1.0, 1.0, 1.0],
+ 'enable_padding': False
}],
remappings=[('encoded_tensor', 'tensor_pub')]
)
diff --git a/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py b/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py
index a3dcf29..510f1f3 100644
--- a/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py
+++ b/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -33,16 +33,37 @@ def generate_launch_description():
with open(labels_file_path, 'r') as fd:
label_list = fd.read().strip().splitlines()
+ image_resize_node_left = ComposableNode(
+ package='isaac_ros_image_proc',
+ plugin='nvidia::isaac_ros::image_proc::ResizeNode',
+ name='image_resize_node_left',
+ parameters=[{
+ 'output_width': 1280,
+ 'output_height': 720,
+ 'encoding_desired': 'rgb8',
+ }],
+ remappings=[
+ ('camera_info', 'front_stereo_camera/left_rgb/camerainfo'),
+ ('image', 'front_stereo_camera/left_rgb/image_raw'),
+ ('resize/camera_info', 'front_stereo_camera/left_rgb/camerainfo_resize'),
+ ('resize/image', 'front_stereo_camera/left_rgb/image_resize')]
+ )
+
encoder_node = ComposableNode(
name='dnn_image_encoder',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
parameters=[{
+ 'input_image_width': 1280,
+ 'input_image_height': 720,
'network_image_width': 1280,
- 'network_image_height': 720
+ 'network_image_height': 720,
+ 'image_mean': [0.0, 0.0, 0.0],
+ 'image_stddev': [1.0, 1.0, 1.0],
+ 'enable_padding': False
}],
remappings=[('encoded_tensor', 'tensor_pub'),
- ('image', 'rgb_left')]
+ ('image', 'front_stereo_camera/left_rgb/image_resize')]
)
triton_node = ComposableNode(
@@ -77,7 +98,7 @@ def generate_launch_description():
package='rclcpp_components',
executable='component_container_mt',
composable_node_descriptions=[
- encoder_node, triton_node, detectnet_decoder_node],
+ image_resize_node_left, encoder_node, triton_node, detectnet_decoder_node],
output='screen'
)
@@ -85,7 +106,7 @@ def generate_launch_description():
package='isaac_ros_detectnet',
executable='isaac_ros_detectnet_visualizer.py',
name='detectnet_visualizer',
- remappings=[('image', 'rgb_left')]
+ remappings=[('image', 'front_stereo_camera/left_rgb/image_resize')]
)
diff --git a/isaac_ros_detectnet/launch/isaac_ros_detectnet_quickstart.launch.py b/isaac_ros_detectnet/launch/isaac_ros_detectnet_quickstart.launch.py
index 108eedd..288cdbd 100644
--- a/isaac_ros_detectnet/launch/isaac_ros_detectnet_quickstart.launch.py
+++ b/isaac_ros_detectnet/launch/isaac_ros_detectnet_quickstart.launch.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
diff --git a/isaac_ros_detectnet/package.xml b/isaac_ros_detectnet/package.xml
index d6ae923..d050388 100644
--- a/isaac_ros_detectnet/package.xml
+++ b/isaac_ros_detectnet/package.xml
@@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0
isaac_ros_detectnet
- 0.31.0
+ 2.0.0DetectNet model processingAshwin Varghese Kuruttukulam
@@ -44,7 +44,7 @@ SPDX-License-Identifier: Apache-2.0
ament_lint_autoament_lint_commonisaac_ros_test
- isaac_ros_dnn_encoders
+ isaac_ros_dnn_image_encoderisaac_ros_triton
diff --git a/isaac_ros_detectnet/resources/peoplenet_config.pbtxt b/isaac_ros_detectnet/resources/peoplenet_config.pbtxt
new file mode 100644
index 0000000..97f32a4
--- /dev/null
+++ b/isaac_ros_detectnet/resources/peoplenet_config.pbtxt
@@ -0,0 +1,29 @@
+name: "peoplenet"
+platform: "tensorrt_plan"
+max_batch_size: 16
+input [
+ {
+ name: "input_1"
+ data_type: TYPE_FP32
+ format: FORMAT_NCHW
+ dims: [ 3, 544, 960 ]
+ }
+]
+output [
+ {
+ name: "output_bbox/BiasAdd"
+ data_type: TYPE_FP32
+ dims: [ 12, 34, 60 ]
+ },
+ {
+ name: "output_cov/Sigmoid"
+ data_type: TYPE_FP32
+ dims: [ 3, 34, 60 ]
+ }
+]
+dynamic_batching { }
+version_policy: {
+ specific {
+ versions: [ 1 ]
+ }
+}
diff --git a/isaac_ros_detectnet/scripts/isaac_ros_detectnet_visualizer.py b/isaac_ros_detectnet/scripts/isaac_ros_detectnet_visualizer.py
index 8203aa2..875e202 100755
--- a/isaac_ros_detectnet/scripts/isaac_ros_detectnet_visualizer.py
+++ b/isaac_ros_detectnet/scripts/isaac_ros_detectnet_visualizer.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
diff --git a/isaac_ros_detectnet/test/isaac_ros_detectnet_pol_test.py b/isaac_ros_detectnet/test/isaac_ros_detectnet_pol_test.py
index 8a5ba5a..05c7f81 100644
--- a/isaac_ros_detectnet/test/isaac_ros_detectnet_pol_test.py
+++ b/isaac_ros_detectnet/test/isaac_ros_detectnet_pol_test.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2022-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -91,11 +91,13 @@ def generate_test_description():
encoder_node = ComposableNode(
name='DnnImageEncoderNode',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
namespace=IsaacROSDetectNetPipelineTest.generate_namespace(
_TEST_CASE_NAMESPACE),
parameters=[{
+ 'input_image_width': 640,
+ 'input_image_height': 368,
'network_image_width': 640,
'network_image_height': 368
}],
@@ -189,6 +191,7 @@ def test_image_detection(self, test_folder):
try:
image = JSONConversion.load_image_from_json(
test_folder / 'detections.json')
+ image.header.stamp = self.node.get_clock().now().to_msg()
ground_truth = open(test_folder.joinpath(
'expected_detections.txt'), 'r')
expected_detections = []
@@ -210,8 +213,6 @@ def test_image_detection(self, test_folder):
rclpy.spin_once(self.node, timeout_sec=0.1)
if 'detectnet/detections' in received_messages:
- pprint(
- received_messages['detectnet/detections'].detections[0])
done = True
break
@@ -220,8 +221,12 @@ def test_image_detection(self, test_folder):
detection_list = received_messages['detectnet/detections'].detections
+ pprint(detection_list)
+
pixel_tolerance = 2.0
+ self.assertGreater(len(detection_list), 0, 'No detections in detection list!')
+
self.assertAlmostEqual(detection_list[0].bbox.size_x,
expected_detections[0]['width'],
None, 'Received incorrect width', pixel_tolerance)
diff --git a/isaac_ros_yolov8/CMakeLists.txt b/isaac_ros_yolov8/CMakeLists.txt
new file mode 100644
index 0000000..206d844
--- /dev/null
+++ b/isaac_ros_yolov8/CMakeLists.txt
@@ -0,0 +1,65 @@
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+cmake_minimum_required(VERSION 3.22.1)
+project(isaac_ros_yolov8 LANGUAGES C CXX)
+
+if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
+ add_compile_options(-Wall -Wextra -Wpedantic)
+endif()
+
+find_package(ament_cmake_auto REQUIRED)
+find_package(rclcpp REQUIRED)
+find_package(sensor_msgs REQUIRED)
+find_package(vision_msgs REQUIRED)
+find_package(geometry_msgs REQUIRED)
+find_package(cv_bridge REQUIRED)
+find_package(OpenCV REQUIRED)
+include_directories(${OpenCV_INCLUDE_DIRS})
+
+ament_auto_find_build_dependencies()
+find_package(ament_cmake_python REQUIRED)
+
+# yolov8 Decoder node
+ament_auto_add_library(yolov8_decoder_node SHARED src/yolov8_decoder_node.cpp)
+rclcpp_components_register_nodes(yolov8_decoder_node "nvidia::isaac_ros::yolov8::YoloV8DecoderNode")
+set(node_plugins "${node_plugins}nvidia::isaac_ros::yolov8::YoloV8DecoderNode;$\n")
+
+target_link_libraries(yolov8_decoder_node ${OpenCV_LIBRARIES})
+
+if(BUILD_TESTING)
+ find_package(ament_lint_auto REQUIRED)
+ ament_lint_auto_find_test_dependencies()
+
+
+ # The FindPythonInterp and FindPythonLibs modules are removed
+ if(POLICY CMP0148)
+ cmake_policy(SET CMP0148 OLD)
+ endif()
+
+ find_package(launch_testing_ament_cmake REQUIRED)
+endif()
+
+# Visualizer python scripts
+ament_python_install_package(${PROJECT_NAME})
+
+install(PROGRAMS
+ scripts/isaac_ros_yolov8_visualizer.py
+ DESTINATION lib/${PROJECT_NAME}
+)
+
+ament_auto_package(INSTALL_TO_SHARE launch)
diff --git a/isaac_ros_yolov8/include/isaac_ros_yolov8/yolov8_decoder_node.hpp b/isaac_ros_yolov8/include/isaac_ros_yolov8/yolov8_decoder_node.hpp
new file mode 100644
index 0000000..b3f586c
--- /dev/null
+++ b/isaac_ros_yolov8/include/isaac_ros_yolov8/yolov8_decoder_node.hpp
@@ -0,0 +1,68 @@
+// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+// Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+#ifndef ISAAC_ROS_YOLOV8__YOLOV8_DECODER_NODE_HPP_
+#define ISAAC_ROS_YOLOV8__YOLOV8_DECODER_NODE_HPP_
+
+#include
+#include
+
+#include "rclcpp/rclcpp.hpp"
+
+#include "isaac_ros_managed_nitros/managed_nitros_subscriber.hpp"
+
+#include "std_msgs/msg/string.hpp"
+#include "vision_msgs/msg/detection2_d_array.hpp"
+#include "isaac_ros_nitros_tensor_list_type/nitros_tensor_list_view.hpp"
+
+namespace nvidia
+{
+namespace isaac_ros
+{
+namespace yolov8
+{
+
+class YoloV8DecoderNode : public rclcpp::Node
+{
+public:
+ explicit YoloV8DecoderNode(const rclcpp::NodeOptions options = rclcpp::NodeOptions());
+
+ ~YoloV8DecoderNode();
+
+private:
+ void InputCallback(const nvidia::isaac_ros::nitros::NitrosTensorListView & msg);
+
+ // Subscription to input NitrosTensorList messages
+ std::shared_ptr> nitros_sub_;
+
+ // Publisher for output Detection2DArray messages
+ rclcpp::Publisher::SharedPtr pub_;
+
+ // Name of tensor in NitrosTensorList
+ std::string tensor_name_{};
+
+ // YOLOv8 Decoder Parameters
+ double confidence_threshold_{};
+ double nms_threshold_{};
+};
+
+} // namespace yolov8
+} // namespace isaac_ros
+} // namespace nvidia
+
+#endif // ISAAC_ROS_YOLOV8__YOLOV8_DECODER_NODE_HPP_
diff --git a/isaac_ros_yolov8/isaac_ros_yolov8/__init__.py b/isaac_ros_yolov8/isaac_ros_yolov8/__init__.py
new file mode 100644
index 0000000..6bbcb13
--- /dev/null
+++ b/isaac_ros_yolov8/isaac_ros_yolov8/__init__.py
@@ -0,0 +1,16 @@
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
diff --git a/isaac_ros_yolov8/launch/isaac_ros_yolov8_visualize.launch.py b/isaac_ros_yolov8/launch/isaac_ros_yolov8_visualize.launch.py
new file mode 100644
index 0000000..4a5419b
--- /dev/null
+++ b/isaac_ros_yolov8/launch/isaac_ros_yolov8_visualize.launch.py
@@ -0,0 +1,46 @@
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+import os
+
+from ament_index_python.packages import get_package_share_directory
+from launch import LaunchDescription
+from launch.actions import IncludeLaunchDescription
+from launch.launch_description_sources import PythonLaunchDescriptionSource
+from launch_ros.actions import Node
+
+
+def generate_launch_description():
+ my_package_dir = get_package_share_directory('isaac_ros_yolov8')
+ return LaunchDescription([
+ IncludeLaunchDescription(
+ PythonLaunchDescriptionSource([os.path.join(
+ my_package_dir, 'launch'),
+ '/yolov8_tensor_rt.launch.py'])
+ ),
+ Node(
+ package='isaac_ros_yolov8',
+ executable='isaac_ros_yolov8_visualizer.py',
+ name='yolov8_visualizer'
+ ),
+ Node(
+ package='rqt_image_view',
+ executable='rqt_image_view',
+ name='image_view',
+ arguments=['/yolov8_processed_image']
+ )
+ ])
diff --git a/isaac_ros_yolov8/launch/yolov8_tensor_rt.launch.py b/isaac_ros_yolov8/launch/yolov8_tensor_rt.launch.py
new file mode 100644
index 0000000..b50a732
--- /dev/null
+++ b/isaac_ros_yolov8/launch/yolov8_tensor_rt.launch.py
@@ -0,0 +1,137 @@
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+import launch
+from launch.actions import DeclareLaunchArgument
+from launch.substitutions import LaunchConfiguration
+from launch_ros.actions import ComposableNodeContainer
+from launch_ros.descriptions import ComposableNode
+
+
+def generate_launch_description():
+ """Generate launch description for TensorRT ROS 2 node."""
+ # By default loads and runs mobilenetv2-1.0 included in isaac_ros_dnn_inference/models
+ launch_args = [
+ DeclareLaunchArgument(
+ 'model_file_path',
+ default_value='',
+ description='The absolute file path to the ONNX file'),
+ DeclareLaunchArgument(
+ 'engine_file_path',
+ default_value='',
+ description='The absolute file path to the TensorRT engine file'),
+ DeclareLaunchArgument(
+ 'input_tensor_names',
+ default_value='["input_tensor"]',
+ description='A list of tensor names to bound to the specified input binding names'),
+ DeclareLaunchArgument(
+ 'input_binding_names',
+ default_value='[""]',
+ description='A list of input tensor binding names (specified by model)'),
+ DeclareLaunchArgument(
+ 'output_tensor_names',
+ default_value='["output_tensor"]',
+ description='A list of tensor names to bound to the specified output binding names'),
+ DeclareLaunchArgument(
+ 'output_binding_names',
+ default_value='[""]',
+ description='A list of output tensor binding names (specified by model)'),
+ DeclareLaunchArgument(
+ 'verbose',
+ default_value='False',
+ description='Whether TensorRT should verbosely log or not'),
+ DeclareLaunchArgument(
+ 'force_engine_update',
+ default_value='False',
+ description='Whether TensorRT should update the TensorRT engine file or not'),
+ ]
+
+ # DNN Image Encoder parameters
+ input_image_width = LaunchConfiguration('input_image_width')
+ input_image_height = LaunchConfiguration('input_image_height')
+ network_image_width = LaunchConfiguration('network_image_width')
+ network_image_height = LaunchConfiguration('network_image_height')
+ image_mean = LaunchConfiguration('image_mean')
+ image_stddev = LaunchConfiguration('image_stddev')
+
+ # TensorRT parameters
+ model_file_path = LaunchConfiguration('model_file_path')
+ engine_file_path = LaunchConfiguration('engine_file_path')
+ input_tensor_names = LaunchConfiguration('input_tensor_names')
+ input_binding_names = LaunchConfiguration('input_binding_names')
+ output_tensor_names = LaunchConfiguration('output_tensor_names')
+ output_binding_names = LaunchConfiguration('output_binding_names')
+ verbose = LaunchConfiguration('verbose')
+ force_engine_update = LaunchConfiguration('force_engine_update')
+
+ # YOLOv8 Decoder parameters
+ confidence_threshold = LaunchConfiguration('confidence_threshold')
+ nms_threshold = LaunchConfiguration('nms_threshold')
+
+ encoder_node = ComposableNode(
+ name='dnn_image_encoder',
+ package='isaac_ros_dnn_image_encoder',
+ plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
+ remappings=[('encoded_tensor', 'tensor_pub')],
+ parameters=[{
+ 'input_image_width': input_image_width,
+ 'input_image_height': input_image_height,
+ 'network_image_width': network_image_width,
+ 'network_image_height': network_image_height,
+ 'image_mean': image_mean,
+ 'image_stddev': image_stddev,
+ }]
+ )
+
+ tensor_rt_node = ComposableNode(
+ name='tensor_rt',
+ package='isaac_ros_tensor_rt',
+ plugin='nvidia::isaac_ros::dnn_inference::TensorRTNode',
+ parameters=[{
+ 'model_file_path': model_file_path,
+ 'engine_file_path': engine_file_path,
+ 'output_binding_names': output_binding_names,
+ 'output_tensor_names': output_tensor_names,
+ 'input_tensor_names': input_tensor_names,
+ 'input_binding_names': input_binding_names,
+ 'verbose': verbose,
+ 'force_engine_update': force_engine_update
+ }]
+ )
+
+ yolov8_decoder_node = ComposableNode(
+ name='yolov8_decoder_node',
+ package='isaac_ros_yolov8',
+ plugin='nvidia::isaac_ros::yolov8::YoloV8DecoderNode',
+ parameters=[{
+ 'confidence_threshold': confidence_threshold,
+ 'nms_threshold': nms_threshold,
+ }]
+ )
+
+ tensor_rt_container = ComposableNodeContainer(
+ name='tensor_rt_container',
+ package='rclcpp_components',
+ executable='component_container_mt',
+ composable_node_descriptions=[encoder_node, tensor_rt_node, yolov8_decoder_node],
+ output='screen',
+ arguments=['--ros-args', '--log-level', 'INFO'],
+ namespace=''
+ )
+
+ final_launch_description = launch_args + [tensor_rt_container]
+ return launch.LaunchDescription(final_launch_description)
diff --git a/isaac_ros_yolov8/launch_commands.md b/isaac_ros_yolov8/launch_commands.md
new file mode 100644
index 0000000..f9eaddf
--- /dev/null
+++ b/isaac_ros_yolov8/launch_commands.md
@@ -0,0 +1,27 @@
+# Custom Nitros YOLOv8
+
+This sample shows how to use your custom model decoder with Isaac ROS Managed Nitros. We consider the task of Object Detection using YOLOv8 with [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference).
+
+### Launch the image publisher in Terminal 1:
+```
+ros2 run image_publisher image_publisher_node --ros-args --remap /image_raw:=/image`
+```
+
+For example (sample image people_cycles.jpg provided in this repo):
+```
+ros2 run image_publisher image_publisher_node people_cycles.jpg --ros-args --remap /image_raw:=/image
+```
+
+### Launch the inference graph in Terminal 2:
+```
+ros2 launch isaac_ros_yolov8 yolov8_tensor_rt.launch.py engine_file_path:=yolov8s_fp16.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 model_file_path:=yolov8s_fp16.onnx force_engine_update:=False image_mean:=[0.485,0.456,0.406] image_stddev:=[0.229,0.224,0.225] input_image_width:=640 input_image_height:=640
+```
+
+Results will be published to `/detections_output` as Detection2DArray messages.
+
+## Visualizing results:
+```
+ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py engine_file_path:=yolov8s_fp16.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 model_file_path:=yolov8s_fp16.onnx force_engine_update:=False image_mean:=[0.485,0.456,0.406] image_stddev:=[0.229,0.224,0.225] input_image_width:=640 input_image_height:=640
+```
+
+An RQT image window will pop up to display resulting bounding boxes on the input image. These output images are published on the `/yolov8_processed_image` topic.
diff --git a/isaac_ros_yolov8/package.xml b/isaac_ros_yolov8/package.xml
new file mode 100644
index 0000000..3986a33
--- /dev/null
+++ b/isaac_ros_yolov8/package.xml
@@ -0,0 +1,61 @@
+
+
+
+
+
+
+ isaac_ros_yolov8
+ 2.0.0
+ Isaac ROS YOLOv8 decoding
+
+ Hemal Shah
+ Apache-2.0
+ https://developer.nvidia.com/isaac-ros-gems/
+ Asawaree Bhide
+
+ ament_cmake_auto
+
+
+ rclcpp
+ rclcpp_components
+ std_msgs
+ sensor_msgs
+ vision_msgs
+ geometry_msgs
+ tf2_geometry_msgs
+ tf2_ros
+ tf2_msgs
+ isaac_ros_nitros
+ isaac_ros_managed_nitros
+ isaac_ros_nitros_interfaces
+ isaac_ros_tensor_list_interfaces
+ python3-opencv
+ libopencv-dev
+ cv_bridge
+
+ isaac_ros_common
+
+ ament_lint_auto
+ ament_lint_common
+ isaac_ros_test
+
+
+ ament_cmake
+
+
diff --git a/isaac_ros_yolov8/scripts/isaac_ros_yolov8_visualizer.py b/isaac_ros_yolov8/scripts/isaac_ros_yolov8_visualizer.py
new file mode 100755
index 0000000..45463d5
--- /dev/null
+++ b/isaac_ros_yolov8/scripts/isaac_ros_yolov8_visualizer.py
@@ -0,0 +1,183 @@
+#!/usr/bin/env python3
+
+# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# This script listens for images and object detections on the image,
+# then renders the output boxes on top of the image and publishes
+# the result as an image message
+
+import cv2
+import cv_bridge
+import message_filters
+import rclpy
+from rclpy.node import Node
+from sensor_msgs.msg import Image
+from vision_msgs.msg import Detection2DArray
+
+names = {
+ 0: 'person',
+ 1: 'bicycle',
+ 2: 'car',
+ 3: 'motorcycle',
+ 4: 'airplane',
+ 5: 'bus',
+ 6: 'train',
+ 7: 'truck',
+ 8: 'boat',
+ 9: 'traffic light',
+ 10: 'fire hydrant',
+ 11: 'stop sign',
+ 12: 'parking meter',
+ 13: 'bench',
+ 14: 'bird',
+ 15: 'cat',
+ 16: 'dog',
+ 17: 'horse',
+ 18: 'sheep',
+ 19: 'cow',
+ 20: 'elephant',
+ 21: 'bear',
+ 22: 'zebra',
+ 23: 'giraffe',
+ 24: 'backpack',
+ 25: 'umbrella',
+ 26: 'handbag',
+ 27: 'tie',
+ 28: 'suitcase',
+ 29: 'frisbee',
+ 30: 'skis',
+ 31: 'snowboard',
+ 32: 'sports ball',
+ 33: 'kite',
+ 34: 'baseball bat',
+ 35: 'baseball glove',
+ 36: 'skateboard',
+ 37: 'surfboard',
+ 38: 'tennis racket',
+ 39: 'bottle',
+ 40: 'wine glass',
+ 41: 'cup',
+ 42: 'fork',
+ 43: 'knife',
+ 44: 'spoon',
+ 45: 'bowl',
+ 46: 'banana',
+ 47: 'apple',
+ 48: 'sandwich',
+ 49: 'orange',
+ 50: 'broccoli',
+ 51: 'carrot',
+ 52: 'hot dog',
+ 53: 'pizza',
+ 54: 'donut',
+ 55: 'cake',
+ 56: 'chair',
+ 57: 'couch',
+ 58: 'potted plant',
+ 59: 'bed',
+ 60: 'dining table',
+ 61: 'toilet',
+ 62: 'tv',
+ 63: 'laptop',
+ 64: 'mouse',
+ 65: 'remote',
+ 66: 'keyboard',
+ 67: 'cell phone',
+ 68: 'microwave',
+ 69: 'oven',
+ 70: 'toaster',
+ 71: 'sink',
+ 72: 'refrigerator',
+ 73: 'book',
+ 74: 'clock',
+ 75: 'vase',
+ 76: 'scissors',
+ 77: 'teddy bear',
+ 78: 'hair drier',
+ 79: 'toothbrush',
+}
+
+
+class Yolov8Visualizer(Node):
+ QUEUE_SIZE = 10
+ color = (0, 255, 0)
+ bbox_thickness = 2
+
+ def __init__(self):
+ super().__init__('yolov8_visualizer')
+ self._bridge = cv_bridge.CvBridge()
+ self._processed_image_pub = self.create_publisher(
+ Image, 'yolov8_processed_image', self.QUEUE_SIZE)
+
+ self._detections_subscription = message_filters.Subscriber(
+ self,
+ Detection2DArray,
+ 'detections_output')
+ self._image_subscription = message_filters.Subscriber(
+ self,
+ Image,
+ 'image')
+
+ self.time_synchronizer = message_filters.TimeSynchronizer(
+ [self._detections_subscription, self._image_subscription],
+ self.QUEUE_SIZE)
+
+ self.time_synchronizer.registerCallback(self.detections_callback)
+
+ def detections_callback(self, detections_msg, img_msg):
+ txt_color = (255, 0, 255)
+ cv2_img = self._bridge.imgmsg_to_cv2(img_msg)
+ for detection in detections_msg.detections:
+ center_x = detection.bbox.center.position.x
+ center_y = detection.bbox.center.position.y
+ width = detection.bbox.size_x
+ height = detection.bbox.size_y
+
+ label = names[int(detection.results[0].hypothesis.class_id)]
+ conf_score = detection.results[0].hypothesis.score
+ label = f'{label} {conf_score:.2f}'
+
+ min_pt = (round(center_x - (width / 2.0)),
+ round(center_y - (height / 2.0)))
+ max_pt = (round(center_x + (width / 2.0)),
+ round(center_y + (height / 2.0)))
+
+ lw = max(round((img_msg.height + img_msg.width) / 2 * 0.003), 2) # line width
+ tf = max(lw - 1, 1) # font thickness
+ # text width, height
+ w, h = cv2.getTextSize(label, 0, fontScale=lw / 3, thickness=tf)[0]
+ outside = min_pt[1] - h >= 3
+
+ cv2.rectangle(cv2_img, min_pt, max_pt,
+ self.color, self.bbox_thickness)
+ cv2.putText(cv2_img, label, (min_pt[0], min_pt[1]-2 if outside else min_pt[1]+h+2),
+ 0, lw / 3, txt_color, thickness=tf, lineType=cv2.LINE_AA)
+
+ processed_img = self._bridge.cv2_to_imgmsg(
+ cv2_img, encoding=img_msg.encoding)
+ self._processed_image_pub.publish(processed_img)
+
+
+def main():
+ rclpy.init()
+ rclpy.spin(Yolov8Visualizer())
+ rclpy.shutdown()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/isaac_ros_yolov8/src/yolov8_decoder_node.cpp b/isaac_ros_yolov8/src/yolov8_decoder_node.cpp
new file mode 100644
index 0000000..5fd4bfe
--- /dev/null
+++ b/isaac_ros_yolov8/src/yolov8_decoder_node.cpp
@@ -0,0 +1,156 @@
+// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
+// Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+#include "isaac_ros_yolov8/yolov8_decoder_node.hpp"
+
+#include
+#include
+#include
+#include
+#include
+
+#include "isaac_ros_nitros_tensor_list_type/nitros_tensor_list_view.hpp"
+#include "isaac_ros_nitros_tensor_list_type/nitros_tensor_list.hpp"
+
+
+#include
+#include
+#include
+
+#include "vision_msgs/msg/detection2_d_array.hpp"
+#include "tf2_geometry_msgs/tf2_geometry_msgs.hpp"
+
+
+namespace nvidia
+{
+namespace isaac_ros
+{
+namespace yolov8
+{
+YoloV8DecoderNode::YoloV8DecoderNode(const rclcpp::NodeOptions options)
+: rclcpp::Node("yolov8_decoder_node", options),
+ nitros_sub_{std::make_shared>(
+ this,
+ "tensor_sub",
+ nvidia::isaac_ros::nitros::nitros_tensor_list_nchw_rgb_f32_t::supported_type_name,
+ std::bind(&YoloV8DecoderNode::InputCallback, this,
+ std::placeholders::_1))},
+ pub_{create_publisher(
+ "detections_output", 50)},
+ tensor_name_{declare_parameter("tensor_name", "output_tensor")},
+ confidence_threshold_{declare_parameter("confidence_threshold", 0.25)},
+ nms_threshold_{declare_parameter("nms_threshold", 0.45)}
+{}
+
+YoloV8DecoderNode::~YoloV8DecoderNode() = default;
+
+void YoloV8DecoderNode::InputCallback(const nvidia::isaac_ros::nitros::NitrosTensorListView & msg)
+{
+ auto tensor = msg.GetNamedTensor(tensor_name_);
+ size_t buffer_size{tensor.GetTensorSize()};
+ std::vector results_vector{};
+ results_vector.resize(buffer_size);
+ cudaMemcpy(results_vector.data(), tensor.GetBuffer(), buffer_size, cudaMemcpyDefault);
+
+ std::vector bboxes;
+ std::vector scores;
+ std::vector indices;
+ std::vector classes;
+
+ // Output dimensions = [1, 84, 8400]
+ int num_classes = 80;
+ int out_dim = 8400;
+ float * results_data = reinterpret_cast(results_vector.data());
+
+ for (int i = 0; i < out_dim; i++) {
+ float x = *(results_data + i);
+ float y = *(results_data + (out_dim * 1) + i);
+ float w = *(results_data + (out_dim * 2) + i);
+ float h = *(results_data + (out_dim * 3) + i);
+
+ float x1 = (x - (0.5 * w));
+ float y1 = (y - (0.5 * h));
+ float width = w;
+ float height = h;
+
+ std::vector conf;
+ for (int j = 0; j < num_classes; j++) {
+ conf.push_back(*(results_data + (out_dim * (4 + j)) + i));
+ }
+
+ std::vector::iterator ind_max_conf;
+ ind_max_conf = std::max_element(std::begin(conf), std::end(conf));
+ int max_index = distance(std::begin(conf), ind_max_conf);
+ float val_max_conf = *max_element(std::begin(conf), std::end(conf));
+
+ bboxes.push_back(cv::Rect(x1, y1, width, height));
+ indices.push_back(i);
+ scores.push_back(val_max_conf);
+ classes.push_back(max_index);
+ }
+
+ RCLCPP_DEBUG(this->get_logger(), "Count of bboxes: %lu", bboxes.size());
+ cv::dnn::NMSBoxes(bboxes, scores, confidence_threshold_, nms_threshold_, indices, 5);
+ RCLCPP_DEBUG(this->get_logger(), "# boxes after NMS: %lu", indices.size());
+
+ vision_msgs::msg::Detection2DArray final_detections_arr;
+
+ for (size_t i = 0; i < indices.size(); i++) {
+ int ind = indices[i];
+ vision_msgs::msg::Detection2D detection;
+
+ geometry_msgs::msg::Pose center;
+ geometry_msgs::msg::Point position;
+ geometry_msgs::msg::Quaternion orientation;
+
+ // 2D object Bbox
+ vision_msgs::msg::BoundingBox2D bbox;
+ float w = bboxes[ind].width;
+ float h = bboxes[ind].height;
+ float x_center = bboxes[ind].x + (0.5 * w);
+ float y_center = bboxes[ind].y + (0.5 * h);
+ detection.bbox.center.position.x = x_center;
+ detection.bbox.center.position.y = y_center;
+ detection.bbox.size_x = w;
+ detection.bbox.size_y = h;
+
+
+ // Class probabilities
+ vision_msgs::msg::ObjectHypothesisWithPose hyp;
+ hyp.hypothesis.class_id = std::to_string(classes.at(ind));
+ hyp.hypothesis.score = scores.at(ind);
+ detection.results.push_back(hyp);
+
+ detection.header.stamp.sec = msg.GetTimestampSeconds();
+ detection.header.stamp.nanosec = msg.GetTimestampNanoseconds();
+
+ final_detections_arr.detections.push_back(detection);
+ }
+
+ final_detections_arr.header.stamp.sec = msg.GetTimestampSeconds();
+ final_detections_arr.header.stamp.nanosec = msg.GetTimestampNanoseconds();
+ pub_->publish(final_detections_arr);
+}
+
+} // namespace yolov8
+} // namespace isaac_ros
+} // namespace nvidia
+
+// Register as component
+#include "rclcpp_components/register_node_macro.hpp"
+RCLCPP_COMPONENTS_REGISTER_NODE(nvidia::isaac_ros::yolov8::YoloV8DecoderNode)
diff --git a/resources/Isaac_sim_change_translate.png b/resources/Isaac_sim_change_translate.png
deleted file mode 100644
index 4dd3538..0000000
--- a/resources/Isaac_sim_change_translate.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:d5579b7472b34174bd57b009224606e195858ab3e2fdae9852f22a3ebcb7e2b4
-size 97417
diff --git a/resources/Isaac_sim_play.png b/resources/Isaac_sim_play.png
deleted file mode 100644
index 6b61501..0000000
--- a/resources/Isaac_sim_play.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:1a28d2d753d439abffbac791d31b64f395e3f4840d9a4352f7d1f08ad2fd5cfd
-size 1173198
diff --git a/resources/header-image.png b/resources/header-image.png
deleted file mode 100644
index 6fd14fd..0000000
--- a/resources/header-image.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:35596fd5fffa6b09c42bbb12d467269fc3e2c9e3ecfb40981f38e0997e6283a8
-size 694968
diff --git a/resources/isaac_ros_object_detection_example.png b/resources/isaac_ros_object_detection_example.png
deleted file mode 100644
index b34da5f..0000000
--- a/resources/isaac_ros_object_detection_example.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e0cfeb34d391b064d6276e9bdd63741318e100c85ef0fe98ca50acb0975b9fb4
-size 81637
diff --git a/resources/isaac_ros_object_detection_example_bbox.png b/resources/isaac_ros_object_detection_example_bbox.png
deleted file mode 100644
index 5d9e4bd..0000000
--- a/resources/isaac_ros_object_detection_example_bbox.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:42cc88a3afab5e35d07f58ad89ffa30992f13b7c248907dd723de3b749962480
-size 80741
diff --git a/resources/isaac_ros_object_detection_example_bboxseg.png b/resources/isaac_ros_object_detection_example_bboxseg.png
deleted file mode 100644
index 86538ab..0000000
--- a/resources/isaac_ros_object_detection_example_bboxseg.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:c92c645a7cf9893e5f6325e69869ebcfc6c728eeed3d56a259437a585d66ce21
-size 75446
diff --git a/resources/isaac_ros_object_detection_nodegraph.png b/resources/isaac_ros_object_detection_nodegraph.png
deleted file mode 100644
index 6f0ec0d..0000000
--- a/resources/isaac_ros_object_detection_nodegraph.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:3c1bf08c0c206bce9d3eaed1f4778abb319c7a5f611fa80477f4947441ceb14c
-size 110779
diff --git a/resources/isaac_sim_detectnet_output.png b/resources/isaac_sim_detectnet_output.png
deleted file mode 100644
index 00eb4da..0000000
--- a/resources/isaac_sim_detectnet_output.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:582c33266eef8b5466b9aa0faa793474ecb2be4b27015a46200543a17ca4a169
-size 1955739
diff --git a/resources/ros2_detectnet_node_setup.svg b/resources/ros2_detectnet_node_setup.svg
deleted file mode 100644
index 488b7f6..0000000
--- a/resources/ros2_detectnet_node_setup.svg
+++ /dev/null
@@ -1 +0,0 @@
-
\ No newline at end of file
diff --git a/resources/rqt_visualizer.png b/resources/rqt_visualizer.png
deleted file mode 100644
index 4bd9dbc..0000000
--- a/resources/rqt_visualizer.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e7246a35666990234500b19660b6a94981d07459d1dee510b63380edb057b429
-size 1341926
diff --git a/resources/tlt_matrix2_updated.png b/resources/tlt_matrix2_updated.png
deleted file mode 100644
index 35e32b5..0000000
--- a/resources/tlt_matrix2_updated.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:63c3224b163e1a01ec6f3e6f6d2e262c715ecd40b88cdb19c52d32aaec4809bd
-size 66217
diff --git a/resources/yolov8s_fp16.onnx b/resources/yolov8s_fp16.onnx
new file mode 100644
index 0000000..4f9c451
--- /dev/null
+++ b/resources/yolov8s_fp16.onnx
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ab61094d96925b1d8318e179c038772df71be0416a49651148989f862880154
+size 44824535
+
