A Control Barrier Function Based Visual Servoing Framework for Safe Operation of UAVs in GNSS Denied Settings using Fiducial Markers
This repository contains the software used to implement the proposed control barrier funciton based UAV, submitted to SSRR24. A video demonstration of the system can be found on the New Dexterity website
Unmanned Aerial Vehicles (UAVs) can offer a versatile solution for the autonomous inspection of a variety of buildings, monuments, or infrastructure. Such inspections may take place regularly or after a sudden event (e.g., earthquake) to understand the state of the point of interest and schedule preventive maintenance or capture and respond to potential damage. But UAVs are often called to operate in complex, dynamic, GNSS denied settings where the only means of navigation is by utilizing appropriate computer vision and perception schemes. In this paper, we propose a Control Barrier Function (CBF) based visual servoing framework to ensure the safe operation of a UAV in a GNSS denied setting using fiducial markers. The CBF maintains view of the marker throughout a mission, which the UAV uses to precisely localise itself in the environment. This continuous localisation ensures the UAV can control itself robustly, as well as help to accurately record information about its surroundings. The methods are experimentally validated with extensive outdoor experiments, inspecting a complex metallic structure at a public park in Auckland, New Zealand.
The 2 main components software of the system are the ROS Noetic docker enviroment where we run our custom ROS nodes, and the ArUco detection script which runs natively on the host system.
The ArUco script is aruco_mqtt_pub.py, and must be run directly on the Raspberry Pi. It is responsible for detecting the ArUco markers in the field of view, and solving the Perspective-n-Point problem for a pair of markers. It then publishes this detected marker pair to the locally hosted MQTT broker so that it can be accessed inside of the Docker enviroment. It also annotates the view and creates a fullscreen window that allows the monitoring of the system remotely using our analog video tranmitter (described in hardware section).
To run a ROS (Robot Operating System) enviroment, we take advantage of the ros:noetic-ros-core docker image. We have created a custom docker file that sources this image and esablishes the envrioemnt needed to run all of our various nodes. This includes installing ROS packages, Python packages, OpenCV, and some other useful tools for debugging such as terminator. The docker file is called dockerfile_noetic and can be used to build an image by running docker build -t ros_drone_noetic -f dockerfile_noetic .
Finally, the visual_servo_drone directory is the custom ROS package containing the 5 nodes as described in the paper, and a 6th logging node which was used to gather the data in a robust manor to analayse many aspects of the system, not just the final output to the flight controller. Placing this directory in the src directory of the ROS workspace will allow any of the nodes to be useds, as well as a launch file to run the entire system. The package also contains 5 custom ROS message descriptions which are used to implement the communication between the nodes.