🏀 Analyze basketball shots and shooting pose with machine learning!

This is an artificial intelligence application built on the concept of object detection. Analyze basketball shots by digging into the data collected from object detection. We can get the result by simply uploading files to the web App, or submitting a POST request to the API. Please check the features below. There are more features coming up! Feel free to follow.

All the data for the shooting pose analysis is calculated by implementing OpenPose. Please note that this is an implementation only for noncommercial research use only. Please read the LICENSE, which is exaclty same as the CMU's OpenPose License.

If your are interested in the concept of human pose estimation, I have written a research paper summary of OpenPose. Check it out!

Getting Started

These instructions will get you a copy of the project up and running on your local machine.

Get a copy

Get a copy of this project by simply running the git clone command.

git clone https://github.com/chonyy/AI-basketball-analysis.git

Prerequisites

Before running the project, we have to install all the dependencies from requirements.txt

pip install -r requirements.txt

Please note that you need a GPU with proper CUDA setup to run the video analysis, since a CUDA device is required to run OpenPose.

Hosting

Last, get the project hosted on your local machine with a single command.

python app.py

Alternatives

Google Colab

Thanks to hardik0. Now we can play around with it without a GPU machine!

Heroku

This project is also hosted on Heroku. However, the heavy computation of TensorFlow may cause Timeout error and crash the app (especially for video analysis). Therefore, hosting the project on your local machine is more preferable.

Please note that the shooting pose analysis won't be running on the Heroku hosted website, since a CUDA device is required to run OpenPose.

Project Structure

Features

This project has three main features, shot analysis, shot detection, detection API.

Shot and Pose analysis

Shot counting

Counting shooting attempts and missing, scoring shots from the input video. Detection keypoints in different colors have different meanings listed below:

• Blue: Detected basketball in normal status
• Purple: Undetermined shot
• Green: Shot went in
• Red: Miss

Pose analysis

Implementing OpenPose to calculate the angle of elbow and knee during shooting.

Release angle and release time are calculated by all the data collected from shot analysis and pose analysis. Please note that there will be a relatively big error for the release time since it was calculated as the total time when the ball is in hand.

Shot detection

Detection will be shown on the image. The confidence and the coordinate of the detection will be listed below.

Detection API

Get the JSON response by submitting a POST request to (./detection_json) with "image" as KEY and input image as VALUE.

Detection model

The object detection model is trained with the Faster R-CNN model architecture, which includes pretrained weight on COCO dataset. Taking the configuration from the model architecture and train it on my own dataset.

Future plans

• Change the model to a more efficient YOLOv4
• Implement SORT Tracking algorithm to filter out false detection
• Improve the visual effect
• Improve the efficiency, making it executable on web app services.