diff --git a/README.md b/README.md index 36e5d59..1929e67 100644 --- a/README.md +++ b/README.md @@ -7,28 +7,35 @@ NVIDIA-accelerated, deep learned model support for object detection including De ## Overview -[Isaac ROS Object Detection](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) contains an ROS 2 package to perform object -detection. `isaac_ros_detectnet` provides a method for spatial +[Isaac ROS Object Detection](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection) contains ROS 2 packages to perform object +detection. +`isaac_ros_rtdetr`, `isaac_ros_detectnet`, and `isaac_ros_yolov8` each provide a method for spatial classification using bounding boxes with an input image. Classification -is performed by a GPU-accelerated -[DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2) -model. The output prediction can be used by perception functions to +is performed by a GPU-accelerated model of the appropriate architecture: + +- `isaac_ros_rtdetr`: [RT-DETR models](https://nvidia-isaac-ros.github.io/concepts/object_detection/rtdetr/index.html) +- `isaac_ros_detectnet`: [DetectNet models](https://nvidia-isaac-ros.github.io/concepts/object_detection/detectnet/index.html) +- `isaac_ros_yolov8`: [YOLOv8 models](https://nvidia-isaac-ros.github.io/concepts/object_detection/yolov8/index.html) + +The output prediction can be used by perception functions to understand the presence and spatial location of an object in an image.
image
-`isaac_ros_detectnet` is used in a graph of nodes to provide a +Each Isaac ROS Object Detection package is used in a graph of nodes to provide a bounding box detection array with object classes from an input image. A -[DetectNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/pretrained_detectnet_v2) -model is required to produce the detection array. Input images may need -to be cropped and resized to maintain the aspect ratio and match the -input resolution of DetectNet; image resolution may be reduced to +trained model of the appropriate architecture is required to produce the detection array. + +Input images may need to be cropped and resized to maintain the aspect ratio and match the +input resolution of the specific object detection model; image resolution may be reduced to improve DNN inference performance, which typically scales directly with the number of pixels in the image. `isaac_ros_dnn_image_encoder` -provides a DNN encoder to process the input image into Tensors for the -DetectNet model. Prediction results are clustered in the DNN decoder to -group multiple detections on the same object. Output is provided as a -detection array with object classes. +provides DNN encoder utilities to process the input image into Tensors for the +object detection models. +Prediction results are decoded in model-specific ways, +often involving clustering and thresholding to group multiple detections +on the same object and reduce spurious detections. +Output is provided as a detection array with object classes. DNNs have a minimum number of pixels that need to be visible on the object to provide a classification prediction. If a person cannot see @@ -41,10 +48,6 @@ image, which provides 1K pixels for the person, and fail to detect the same person in the downscaled resolution, which only provides 0.25K pixels for the person. -> [!Note] -> DetectNet is similar to other popular object detection -> models such as YOLOV3, FasterRCNN, and SSD, while being efficient at -> detecting multiple object classes in large images.
image
Object detection classifies a rectangle of pixels as containing an diff --git a/gxf_isaac_detectnet/package.xml b/gxf_isaac_detectnet/package.xml index 61f365e..0d2dd31 100644 --- a/gxf_isaac_detectnet/package.xml +++ b/gxf_isaac_detectnet/package.xml @@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0 gxf_isaac_detectnet - 3.0.0 + 3.0.1 Detectnet GXF extension. Isaac ROS Maintainers diff --git a/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py b/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py index a2b7225..1ce8d7e 100644 --- a/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py +++ b/isaac_ros_detectnet/launch/isaac_ros_detectnet_isaac_sim.launch.py @@ -48,10 +48,10 @@ def generate_launch_description(): 'encoding_desired': 'rgb8', }], remappings=[ - ('camera_info', 'front_stereo_camera/left_rgb/camerainfo'), - ('image', 'front_stereo_camera/left_rgb/image_raw'), - ('resize/camera_info', 'front_stereo_camera/left_rgb/camerainfo_resize'), - ('resize/image', 'front_stereo_camera/left_rgb/image_resize')] + ('camera_info', 'front_stereo_camera/left/camera_info'), + ('image', 'front_stereo_camera/left/image_rect_color'), + ('resize/camera_info', 'front_stereo_camera/left/camera_info_resize'), + ('resize/image', 'front_stereo_camera/left/image_resize')] ) encoder_dir = get_package_share_directory('isaac_ros_dnn_image_encoder') @@ -71,8 +71,8 @@ def generate_launch_description(): 'attach_to_shared_component_container': 'True', 'component_container_name': 'detectnet_container/detectnet_container', 'dnn_image_encoder_namespace': 'detectnet_encoder', - 'image_input_topic': '/front_stereo_camera/left_rgb/image_resize', - 'camera_info_input_topic': '/front_stereo_camera/left_rgb/camerainfo_resize', + 'image_input_topic': '/front_stereo_camera/left/image_resize', + 'camera_info_input_topic': '/front_stereo_camera/left/camera_info_resize', 'tensor_output_topic': '/tensor_pub', }.items(), ) @@ -117,7 +117,7 @@ def generate_launch_description(): package='isaac_ros_detectnet', executable='isaac_ros_detectnet_visualizer.py', name='detectnet_visualizer', - remappings=[('image', 'front_stereo_camera/left_rgb/image_resize')] + remappings=[('image', 'front_stereo_camera/left/image_resize')] ) diff --git a/isaac_ros_detectnet/package.xml b/isaac_ros_detectnet/package.xml index a7b1af4..d346a01 100644 --- a/isaac_ros_detectnet/package.xml +++ b/isaac_ros_detectnet/package.xml @@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0 isaac_ros_detectnet - 3.0.0 + 3.0.1 DetectNet model processing Isaac ROS Maintainers diff --git a/isaac_ros_detectnet/scripts/setup_model.sh b/isaac_ros_detectnet/scripts/setup_model.sh index f291cdb..756f78a 100755 --- a/isaac_ros_detectnet/scripts/setup_model.sh +++ b/isaac_ros_detectnet/scripts/setup_model.sh @@ -74,7 +74,7 @@ function setup_model() { model_name_from_model_link=$(extract_model_name "$MODEL_LINK") echo "Model name from model link: $model_name_from_model_link" echo Creating Directory : ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link/1 - rm -rf ${ISAAC_ROS_WS}/isaac_ros_assets/models + rm -rf ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link/1 cd ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link/1 echo Downloading .etlt file from $MODEL_LINK @@ -99,7 +99,6 @@ function setup_model() { -o $OUTPUT_LAYERS\ $MODEL_FILE_NAME echo Copying .pbtxt config file to ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link - cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet export ISAAC_ROS_DETECTNET_PATH=$(ros2 pkg prefix isaac_ros_detectnet --share) cp $ISAAC_ROS_DETECTNET_PATH/config/$CONFIG_FILE \ ${ISAAC_ROS_WS}/isaac_ros_assets/models/$model_name_from_model_link/config.pbtxt diff --git a/isaac_ros_rtdetr/launch/isaac_ros_rtdetr_isaac_sim.launch.py b/isaac_ros_rtdetr/launch/isaac_ros_rtdetr_isaac_sim.launch.py index 8ab537a..28de1df 100644 --- a/isaac_ros_rtdetr/launch/isaac_ros_rtdetr_isaac_sim.launch.py +++ b/isaac_ros_rtdetr/launch/isaac_ros_rtdetr_isaac_sim.launch.py @@ -79,8 +79,8 @@ def generate_launch_description(): 'disable_padding': True }], remappings=[ - ('image', 'front_stereo_camera/left_rgb/image_raw'), - ('camera_info', 'front_stereo_camera/left_rgb/camerainfo') + ('image', 'front_stereo_camera/left/image_rect_color'), + ('camera_info', 'front_stereo_camera/left/camera_info') ] ) pad_node = ComposableNode( diff --git a/isaac_ros_rtdetr/package.xml b/isaac_ros_rtdetr/package.xml index 11e8ebe..61cc802 100644 --- a/isaac_ros_rtdetr/package.xml +++ b/isaac_ros_rtdetr/package.xml @@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0 isaac_ros_rtdetr - 3.0.0 + 3.0.1 RT-DETR model processing Isaac ROS Maintainers diff --git a/isaac_ros_yolov8/package.xml b/isaac_ros_yolov8/package.xml index cbc0e68..acc8959 100644 --- a/isaac_ros_yolov8/package.xml +++ b/isaac_ros_yolov8/package.xml @@ -21,7 +21,7 @@ isaac_ros_yolov8 - 3.0.0 + 3.0.1 Isaac ROS YOLOv8 decoding Isaac ROS Maintainers