NASA Challenge_[@pojenwang]_[Space-ROS-NVIDIA-Isaac-Sim Curiosity's Sulfur Stone Discovery]

isaac_sim/.pre-commit-config.yaml

@@ -0,0 +1,44 @@
+default_language_version:
+  python: python3.10
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.6.0
+    hooks:
+      - id: check-ast
+      - id: check-docstring-first
+      - id: check-executables-have-shebangs
+      - id: check-shebang-scripts-are-executable
+      - id: check-symlinks
+      - id: check-toml
+      - id: check-yaml
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
+    rev: v2.13.0
+    hooks:
+      - id: pretty-format-toml
+        args: ["--autofix"]
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.4.4
+    hooks:
+      - id: ruff
+        args: ["--fix", "--unsafe-fixes", "--config", "ruff.toml"]
+      - id: ruff-format
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.10.0
+    hooks:
+      - id: mypy
+        args:
+          [
+            "--allow-subclassing-any",
+            "--strict",
+            "--ignore-missing-imports",
+            "--scripts-are-modules",
+            "--install-types",
+            "--non-interactive",
+          ]
+  - repo: https://github.com/AleksaC/hadolint-py
+    rev: v2.12.1b3
+    hooks:
+      - id: hadolint
+        args: [--ignore, DL3008, --ignore, DL4006] # TODO: Pin apt-versions and remove DL3008 from ignore list

isaac_sim/README.md

@@ -0,0 +1,197 @@
+![space-ros_isaac-sim_front](resources/rover/repo_front.png)
+# Space ROS Isaac Sim: Curiosity's Sulfur Stone Discovery in the Gediz Vallis Channel Demo
+
+This repository provides a Docker image for our **NASA's Curiosity rover** demo based on **NVidia Isaac Sim** that interfaces with **Space ROS**. We also built a digital twin of the **Gediz Vallis Channel Environment**, focusing on the area surrounding the recent event of **sulfur stone discovery**. The map was developed using the **HIRISE Mars DTM** (real Mars terrain data) and with custom-made rock assets that seamlessly blend into the environment. It provides a highly detailed replication of the environment for both current and future missions. As of September 2024, Curiosity is still exploring within this region.
+
+See [Map Documentation](./docs/maps.md)
+
+## Highlight of the features
+- Docker image of NVIDIA Isaac Sim with Space ROS interface
+- Digital twin environment of the Gediz Vallis channel sulfur stone discovery site with enhenced terrain features
+- Curiosity rover asset integration into the simulation
+- ROS2 interfaces for controlling the rover and sensor feedback
+
+## Advantages of NVidia Isaac Sim
+- Realistic graphic with Ray-Tracing technology
+- PhysX engine for advanced physics simulation
+- Built-in DLSS Frame Generation for improved performence
+- Direct VR/AR support (with extension), useful in planetory exploration simulations
+
+## System Requirements
+
+To run the Curiosity rover simulation in Isaac Sim, ensure your system meets the following requirements:
+
+| Requirement  | Description                                  |
+| ------------ | -------------------------------------------- |
+| OS           | Ubuntu 22.04                                 |
+| GPU          | Nvidia RTX 30XX series or higher             |
+| RAM          | 16 GB or higher                              |
+| CPU          | Intel i7 or higher                           |
+| NVIDIA Driver| ver. 560.35.03 (exact ver. recommended)      |
+
+For more detailed hardware requirements, refer to the [Isaac Sim Hardware Documentation](https://docs.omniverse.nvidia.com/isaacsim/latest/installation/requirements.html).
+
+## Prerequisites
+
+The following software dependencies must be installed on your system:
+
+| Dependency         | Description                                                                                                                                 |
+| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- |
+| Docker Engine      | [Docker Installation Guide](https://docs.docker.com/engine/install/)                                                                        |
+| NVIDIA Toolkit     | [NVIDIA Container Toolkit Installation](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)               |
+
+## Required Docker Images
+
+Ensure the following Docker images are built and available on your system before running the simulation:
+
+| Image               | Description                                                                                                                                                        |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| [space-robots](https://github.com/space-ros/docker/tree/main/space_robots) | The base Docker image for space robotics research.                                           |
+| [moveit2](https://github.com/space-ros/docker/tree/main/moveit2)         | Provides MoveIt2 for motion planning.                                                            |
+| [space ros](https://github.com/space-ros/space-ros)                      | Core ROS2 components for space robotics.                                                                        |
+
+## Building and Running the Container
+
+Start the Docker with
+```
+sudo systemctl start docker
+```
+
+To build and launch the container you should cd to `docker` folder:
+
+To build image use this script:
+```bash
+./build.sh
+```
+
+To run docker container use this script:
+```bash
+./run.sh
+```
+That's all you need to start the simulation environment.
+
+### Cloning Repositories into Docker
+
+The following repositories are cloned into the Docker container during the build process:
+
+1. **[isaac_ros2_utils](https://github.com/AlexanderRex/isaac_ros2_utils)**
+2. **[curiosity_rover_description](https://github.com/AlexanderRex/curiosity_rover_description)**
+3. **[mars_rover_control](https://github.com/AlexanderRex/mars_rover_control)**
+
+## Recommended Development Setup
+
+For an optimized development workflow, it is recommended to use **Visual Studio Code (VSCode)** along with its Docker-related extensions. These tools greatly simplify the process of interacting with Docker containers, especially when debugging and managing development environments.
+
+### Recommended VSCode Extensions:
+- **[Dev Containers](https://code.visualstudio.com/docs/devcontainers/containers)**: Enables you to develop inside a Docker container.
+- **[Docker Extension for VSCode](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker)**: Simplifies the management of Docker containers directly from VSCode.
+
+By using these tools, you can streamline your workflow and improve the experience of working within a Dockerized development environment.
+
+## Usage
+
+IMPORTANT: Before launching scripts, execute this command to attach to **space_ros_isaac_sim** container or use **VSCode remote connection**.
+```bash
+sudo docker exec -it space_ros_isaac_sim /bin/bash
+```
+
+In the first terminal, enter the Docker container(if not already) and launch Isaac Sim with one of the following two environments
+
+- For spawning Curiosity in the sulfur stone area in the **Gediz Vallis channel** environment
+   ```
+   python ~/curiosity_sim/scripts/gale_crater_scene.py 
+   ```
+- For spawning Curiosity in the **test ground** (under development) environment
+   ```
+   python ~/curiosity_sim/scripts/test_scene.py
+   ```
+Note: When launching Isaac Sim for the first time it can take several minutes because of shader compilation.
+
+
+In the second terminal, enter the Docker container and launch the control system:
+```bash
+ros2 launch mars_rover_control mars_rover_control.launch.py
+```
+
+In the third terminal, for manual control using the keyboard:
+```bash
+ros2 run teleop_twist_keyboard teleop_twist_keyboard
+```
+Note: this terminal needs to be focused during keyboard control
+
+### ROS Service Commands
+
+In addition to ROS2 topics, the control graph supports various ROS2 service commands, allowing for precise control of different parts of the rover:
+
+- **Drive the rover forward**:
+    ```bash
+    ros2 service call /move_forward std_srvs/srv/Empty
+    ```
+
+- **Stop the rover**:
+    ```bash
+    ros2 service call /move_stop std_srvs/srv/Empty
+    ```
+
+- **Turn left**:
+    ```bash
+    ros2 service call /turn_left std_srvs/srv/Empty
+    ```
+
+- **Turn right**:
+    ```bash
+    ros2 service call /turn_right std_srvs/srv/Empty
+    ```
+
+- **Open the tool arm**:
+    ```bash
+    ros2 service call /open_arm std_srvs/srv/Empty
+    ```
+
+- **Close the tool arm**:
+    ```bash
+    ros2 service call /close_arm std_srvs/srv/Empty
+    ```
+
+- **Open the mast (camera arm)**:
+    ```bash
+    ros2 service call /mast_open std_srvs/srv/Empty
+    ```
+
+- **Close the mast (camera arm)**:
+    ```bash
+    ros2 service call /mast_close std_srvs/srv/Empty
+    ```
+
+### Manuevering Curiosity on a Slope
+There are many hills in the Gediz Vallis channel environment. Curiosity may drift downhill on a slope and it will require increased speed to counteract the incline.
+
+### Recommended Render Settings for Isaac Sim
+
+To optimize performance and maintain high FPS in the simulation, it is recommended to adjust the render settings in **Isaac Sim** as follows:
+
+- **Enable DLSS**: Deep Learning Super Sampling (DLSS) is a technology that uses AI to upscale images, which can significantly boost performance without compromising visual quality. Enabling DLSS is especially useful for maintaining high FPS in complex scenes.
+  - For more information, refer to the [NVIDIA DLSS documentation](https://docs.omniverse.nvidia.com/materials-and-rendering/latest/rtx-renderer_rt.html#dlss).
+
+- **Adjust Viewport Resolution**: If your system struggles to maintain a stable FPS, reducing the resolution of the **Viewport** window can help. Lowering the viewport resolution reduces the computational load, improving performance during simulation.
+  - You can find more details on adjusting the viewport resolution in the [Viewport Settings documentation](https://docs.omniverse.nvidia.com/extensions/latest/ext_core/ext_viewport/controls/settings.html).
+
+By using these settings, you can balance visual quality with performance, ensuring smoother operation of the simulation even on lower-end systems.
+
+
+
+## Detailed Documentation
+
+The following table provides links to detailed documentation on various components of the **Curiosity Mars Rover** simulation, covering control, sensors, and more.
+
+| Document                  | Description                                                                                  |
+|---------------------------|----------------------------------------------------------------------------------------------|
+| [Maps](./docs/maps.md)                             | Information on maps and environments used in the simulation, providing context for different scenarios.|
+| [Curiosity Description](./docs/curiosity_description.md) | Description and overview of the Curiosity rover model used in the simulation.|
+| [Curiosity Control](./docs/curiosity_control.md)   | Detailed documentation on the control system of the Curiosity rover using ROS2 and action graphs.     |
+| [Curiosity Sensors](./docs/curiosity_sensors.md)   | Explanation of the sensors used in the simulation, including Lidar, camera, and odometry configurations. |
+```
+Challenge Name: NASA Space ROS Sim Summer Sprint Challenge
+Team Lead Freelancer User Name: pojenwang
+Submission Title: Space-ROS-NVIDIA-Isaac-Sim Curiosity's Sulfur Stone Discovery
+```

isaac_sim/docker/Dockerfile

@@ -0,0 +1,82 @@
+# hadolint ignore=DL3007
+FROM osrf/space-ros:latest AS base
+
+# Define arguments used in the metadata definition
+ARG VCS_REF
+ARG VERSION="preview"
+
+# Specify the docker image metadata
+LABEL org.label-schema.schema-version="1.0"
+LABEL org.label-schema.name="Space-ROS-NVIDIA-Isaac-Sim Curiosity's Sulfur Stone Discovery"
+LABEL org.label-schema.description="Repository provides a Docker image for NASA's Curiosity rover demo"
+LABEL org.label-schema.vendor="Open Robotics"
+LABEL org.label-schema.version=${VERSION}
+LABEL org.label-schema.url="https://github.com/space-ros"
+LABEL org.label-schema.vcs-url="https://github.com/space-ros/docker"
+LABEL org.label-schema.vcs-ref=${VCS_REF}
+
+FROM base
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+USER root
+
+COPY --link apt-requirements.txt /apt-requirements.txt
+# hadolint ignore=SC2046
+RUN apt-get update && \
+    apt-get install --no-install-recommends -y $(cat /apt-requirements.txt) && \
+    rm -rf /var/lib/apt/lists/*
+RUN rm /apt-requirements.txt
+
+COPY --link ros-requirements.txt /ros-requirements.txt
+# hadolint ignore=SC2046
+RUN apt-get update && \
+    apt-get install --no-install-recommends -y $(cat /ros-requirements.txt) && \
+    rm -rf /var/lib/apt/lists/* && \
+    rm /ros-requirements.txt
+
+RUN locale-gen en_US en_US.UTF-8 &&  update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
+
+ENV LANG=en_US.UTF-8
+
+# Install Isaac Sim
+COPY --link nvidia-pip-requirements.txt /requirements.txt
+RUN pip install --timeout=600 --no-cache-dir -r /requirements.txt --extra-index-url https://pypi.nvidia.com && \
+    rm /requirements.txt
+
+RUN adduser $USERNAME sudo && echo "$USERNAME ALL=NOPASSWD: ALL" >> /etc/sudoers.d/$USERNAME
+
+RUN mkdir -p /usr/local/lib/python3.10/dist-packages/isaacsim/extscache/omni.gpu_foundation/cache && \
+    mkdir -p /usr/local/lib/python3.10/dist-packages/omni/logs && \
+    mkdir -p /usr/local/lib/python3.10/dist-packages/omni/cache/Kit/106.0/fcefe91f/PhysXLocalMeshCache && \
+    mkdir -p /usr/local/lib/python3.10/dist-packages/omni/data/documents/Kit/apps/Isaac-Sim/scripts/new_stage && \
+    mkdir -p /usr/local/lib/python3.10/dist-packages/omni/data/documents/Kit/shared/screenshots && \
+    chown -R $USERNAME:$USERNAME /usr/local/lib/python3.10/dist-packages/isaacsim/extscache/omni.gpu_foundation/cache && \
+    chown -R $USERNAME:$USERNAME /usr/local/lib/python3.10/dist-packages/omni/logs && \
+    chown -R $USERNAME:$USERNAME /usr/local/lib/python3.10/dist-packages/omni/cache && \
+    chown -R $USERNAME:$USERNAME /usr/local/lib/python3.10/dist-packages/omni/data/documents && \
+    chmod -R 777 /usr/local/lib/python3.10/dist-packages/isaacsim/* && \
+    chmod -R 777 /usr/local/lib/python3.10/dist-packages/omni/logs && \
+    chmod -R 777 /usr/local/lib/python3.10/dist-packages/omni/cache && \
+    chmod -R 777 /usr/local/lib/python3.10/dist-packages/omni/data/documents
+
+USER ${USERNAME}
+
+# Copy the dependency.repos
+COPY --chown=${USERNAME}:${USERNAME} dependency.repos /tmp/dependency.repos
+
+RUN /bin/bash -c "source /opt/ros/humble/setup.bash && \
+    mkdir -p /home/$USERNAME/isaac_ws/src && \
+    cd /home/$USERNAME/isaac_ws && \
+    vcs import src < /tmp/dependency.repos --recursive && \
+    colcon build && \
+    rm /tmp/dependency.repos && \
+    cd /home/$USERNAME && git clone https://github.com/pojenwang/curiosity_sim.git"
+
+# Set environment in bashrc for future sessions
+RUN echo "source /opt/ros/humble/setup.bash" >> /home/"$USERNAME"/.bashrc && \
+    echo "source /home/$USERNAME/isaac_ws/install/setup.bash" >> /home/"$USERNAME"/.bashrc && \
+    echo "export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp" >> /home/"$USERNAME"/.bashrc
+
+COPY --link entrypoint.sh /.entrypoint.sh
+ENTRYPOINT ["/.entrypoint.sh"]

isaac_sim/docker/apt-requirements.txt

@@ -0,0 +1,31 @@
+apt-utils
+bash-completion
+build-essential
+curl
+debian-archive-keyring
+debian-keyring
+gedit
+git
+git-lfs
+gpg-agent
+libxcb-cursor0
+libxcb-xinerama0
+libxkbcommon-x11-0
+locales
+lsb-release
+python-is-python3
+python3
+python3-pip
+python3-setuptools
+software-properties-common
+sudo
+telnet
+tree
+wget
+libglu1-mesa
+libgl1-mesa-glx
+libxrender1
+libcanberra-gtk3-0
+libpci-dev
+xauth
+xterm

isaac_sim/docker/build.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+sudo xhost +
+
+declare SCRIPT_NAME=$(readlink -f ${BASH_SOURCE[0]})
+cd $(dirname $SCRIPT_NAME)
+
+docker compose build space_ros_isaac_sim

isaac_sim/docker/dependency.repos

@@ -0,0 +1,17 @@
+repositories:
+  isaac_ros2_utils:
+    type: git
+    url: https://github.com/AlexanderRex/isaac_ros2_utils.git
+    version: main
+  curiosity_rover_description:
+    type: git
+    url: https://github.com/AlexanderRex/curiosity_rover_description.git
+    version: main
+  mars_rover_control:
+    type: git
+    url: https://github.com/AlexanderRex/mars_rover_control.git
+    version: main
+  ros2_controllers:
+    type: git
+    url: https://github.com/tonylitianyu/ros2_controllers.git
+    version: effort_group_position_controller_2