Skip to content

Latest commit

 

History

History
175 lines (134 loc) · 8.16 KB

README.md

File metadata and controls

175 lines (134 loc) · 8.16 KB
Raw

RoboGuard! Autonomous Security Guard


Logo

MAE148 Final Project

Team 2 SS02 2024

⚠️ The added ROS2 files may not be the latest version we ran on our final project. These are from my laptop but don't show the final changes we made directly from the docker container. The files will be updated when we get access to a Jetson again.

image

Table of Contents
  1. Team Members
  2. Final Project
  3. Robot Design
  4. Acknowledgments
  5. Contact

Team Members

Matthew Huntley, Donovan Sanders, Conor Norris

Team Member Major and Class

  • Matt - Electrical Engineering, Computer System Design - Class of 2025
  • Donovan - Computer Engineering - Class of 2023
  • Conor - Mechanical Engineering, Ctrls & Robotics - Class of 2024

Final Project

Our project goal was to develop a prototype of an autonomous security guard that patrols around a chosen perimeter and detects any people in its path. Then it checks to see if the visitor's face is on the preauthorized guest list and takes action against them if not. We developed our own ROS2 packages as well as integrating the lane detection package that came with the Robocar framework, allowing our vehicle to perform multiple complex tasks simultaneously.

Original Goals

  • Set and follow path given patrol path
  • If a person is recognized while on the patrol path, would it will drive to the person and command them to show their face
  • The user's face will be checked against a facial recognition whitelist. If they are not on the approved list, an alarm is set off. Also, if they do not show their face to the robot within a grace period the alarm is set off.

Goals We Met

  • Robocar Lane Guidance integration with a new pause service using SetBool Service type to follow yellow line path
  • Custom Roboflow model to recognize humans as well as three individuals which are authorized and unauthorized running on Oak-D lite
  • Roboflow ROS2 node with detection logic
  • Custom audio_player topic with subscriber node
  • Using Two Oak-D Cameras to run two different machine learning models: one for Lane Detection and Following and another for Person identification.

Our Hopes and Dreams

Stretch Goal 1

  • Firing Nerf gun at people after unsuccessful identity verification
    • We were unable to get access to a powerful enough servo until the day of presentation, but otherwise accomplished this

Stretch Goal 2

  • Follow unauthorized individuals by publishing to /cmd_vel
    • We didn't have enough time to implement person following and instead it drives on the path while patrolling then stops when identifying and resumes it's patrol path after.

Robot Design

CAD Parts

Final Assembly

Custom Designed Parts

Part CAD Model
OAKD Camera Mount
OAKD Articulating Arm
OAKD Plate Mount
GPS Lidar Mount
Nerf Gun Mount
Perforated Plate

Open Source Parts

Part CAD Model
Jetson Nano Case

Electronic Hardware

Below is a circuit diagram of the electronic hardware setup for the car.

Software

Embedded Systems

Embedded Linux: To use the Jetson nano, we had to connect via ssh so that we could run, upload, and program our robot. The provided Docker container from Dominic Nightengale had ROS2 installed so that we could easily start immediate development on our robot.

Roboflow

  • Implemented Custom Roboflow 3.0 Object Detection Model to detect “Human-person, Conor, Donovan, Matt” which were used to create authorized and unauthorized users for the model to detect. Our model runs “on the edge of the edge” through the use of the Oak-D Lite and Nvidia Jetson Nano.
  • Created basic Image Collection Script for the Oak-D lite to take training images for our model as this was how our robot would “see” in the real environment.

Docker and ROS2 Implementation:

The ROS2 Docker Image provided by Dominic Nightengale has several ROS2 packages that are already implemented and testable on our robot.

  • Used Robocar Lane Navigation Package 2 which makes use of the Lane Guidance node to follow a yellow line. Using this package and integrating a pause service on top of the provided code as well logic for Roboflow Model Detection and Custom Audio Subscriber Node.
  • Created Custom Roboflow Package to use our model for person identification and used detections to publish to other ROS2 nodes.

Challenges

We were initially hoping to use a custom Yolo V8 which ran locally using the Oak-D lite, however, even after much trial and error, we could not get it to run due to an invalid architecture error. We tried several different versions of our model such as the blob and onnx files, however, even after installing Ultralytics to the Docker image (to use our Yolo model), the model would still not run. Went ended up working and being faster in the end was to use the Roboflow 3.0 Object Detection Model available on the website which ported the correct version of our custom Roboflow model over to the Oak-D lite.

Acknowledgments

Thanks to Eric, Alex, and Professor Silberman

Contact