Name	Name	Last commit message	Last commit date
parent directory ..
exe_v2h	exe_v2h
exe_v2l	exe_v2l
images	images
src	src
README.md	README.md
gitattributes	gitattributes

Object Counter Application

Application: Overview

The Object Counter Application is a user-friendly and efficient generic software tool that can be used to create custom counting applications for any scenario. This application uses the advanced YOLOV3/Tiny YOLOv3 algorithm to identify and count objects in images or videos.

Use Cases

The Generic Counter Application is a powerful tool that can be used to count objects in a variety of settings, including:

Animal Counting: The application can be fine tuned to count the animals only. This application can be used for zoo or farm monitoring, also could be used to prevent the road hazards due to animal interference. The list of animals on which the AI model is trained is available in animal_class.txt
Vehicle Counting: The application can be fine tuned to count the vehicle instances per frame. This application can then be used for traffic monitoring at government/corporate buildings.The list of vehicles on which the AI model is trained is available in vehicle_class.txt
General Counting: The general counting applications can be used to count any type of object, from people and cars to inventory and products. They are often used in businesses to track customer traffic, inventory levels, and employee productivity. The list of objects on which the AI model is trained is available in coco_class.txt

The other use cases could be:

Manufacturing: The application can be used to count parts on a production line or to measure the output of a machine.
Retail: The application can be used to count products on a shelf or to track the number of customers in a store.
Safety: The application can be used to count people in a crowd or to monitor the traffic flow in a city.

Key Features

Here are some of the key features of the Generic Counter Application:

Automatic Object Detection: The application utilizes YOLOv3/Tiny YOLOv3 model for detection, identifying and localizing objects specified within the provided frame.
Flexible: The application can be customized to meet the specific needs of any counting scenario.
Customizable Settings: Users can adjust the detection and classification parameters by using the config file provided in the repository.

It has following camera input modes.

Mode	RZ/V2L	RZ/V2H
MIPI Camera	Supported	-
USB Camera	Supported	Supported

Users can select detection target from following list

Animal
Vehicle
General (COCO dataset)

Supported Product

RZ/V2L Evaluation Board Kit (RZ/V2L EVK)
RZ/V2H Evaluation Board Kit (RZ/V2H EVK)

Demo

Application: Requirements

Hardware Requirements

For	Equipment	Details
RZ/V2L	RZ/V2L EVK	Evaluation Board Kit for RZ/V2L. Includes followings. MIPI Camera Module(Google Coral Camera) MicroUSB to Serial Cable for serial communication.
	AC Adapter	USB Power Delivery adapter for the board power supply.
	MicroHDMI Cable	Used to connect the HDMI Monitor and the board. RZ/V2L EVK has microHDMI port.
RZ/V2H	RZ/V2H EVK	Evaluation Board Kit for RZ/V2H.
	AC Adapter	USB Power Delivery adapter for the board power supply. 100W is required.
	HDMI Cable	Used to connect the HDMI Monitor and the board. RZ/V2H EVK has HDMI port.
	USB Camera	Used as a camera input source.
Common	USB Cable Type-C	Connect AC adapter and the board.
	HDMI Monitor	Used to display the graphics of the board.
	microSD card	Used as the filesystem. Must have over 4GB capacity of blank space. Operating Environment: Transcend UHS-I microSD 300S 16GB
	Linux PC	Used to build application and setup microSD card. Operating Environment: Ubuntu 20.04
	SD card reader	Used for setting up microSD card.
	USB Hub	Used to connect USB Keyboard and USB Mouse to the board.
	USB Keyboard	Used to type strings on the terminal of board.
	USB Mouse	Used to operate the mouse on the screen of board.

Note: All external devices will be attached to the board and does not require any driver installation (Plug n Play Type)

Connect the hardware as shown below.

RZ/V2L EVK	RZ/V2H EVK

Note 1: When using the keyboard connected to RZ/V Evaluation Board, the keyboard layout and language are fixed to English.
Note 2: For RZ/V2H EVK, there are USB 2.0 and USB 3.0 ports.
USB camera needs to be connected to appropriate port based on its requirement.

Application: Build Stage

Note: User can skip to the next stage (deploy) if they do not want to build the application.
All pre-built binaries are provided.

Prerequisites

This section expects the user to have completed Step 5 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

AI SDK setup is done.
Following docker container is running on the host machine.

Board Docker container

RZ/V2L EVK rzv2l_ai_sdk_container

RZ/V2H EVK rzv2h_ai_sdk_container

Note: Docker environment is required for building the sample application.

Board	Docker container
RZ/V2L EVK	`rzv2l_ai_sdk_container`
RZ/V2H EVK	`rzv2h_ai_sdk_container`

Application File Generation

On your host machine, copy the repository from the GitHub to the desired location.
1. It is recommended to copy/clone the repository on the data folder, which is mounted on the Docker container.
```
cd <path_to_data_folder_on_host>/data
git clone https://github.com/renesas-rz/rzv_ai_sdk.git
```
Note: This command will download the whole repository, which include all other applications.
If you have already downloaded the repository of the same version, you may not need to run this command.
Run (or start) the docker container and open the bash terminal on the container.
E.g., for RZ/V2L, use the rzv2l_ai_sdk_container as the name of container created from rzv2l_ai_sdk_image docker image.

Note that all the build steps/commands listed below are executed on the docker container bash terminal.
Set your clone directory to the environment variable.
```
export PROJECT_PATH=/drp-ai_tvm/data/rzv_ai_sdk
```
Go to the application source code directory.
```
cd ${PROJECT_PATH}/Q08_object_counter/src
```
Create and move to the build directory.
```
mkdir -p build && cd build
```

Build the application by following the commands below.
For RZ/V2L

cmake -DCMAKE_TOOLCHAIN_FILE=./toolchain/runtime.cmake ..
make -j$(nproc)

For RZ/V2H

cmake -DCMAKE_TOOLCHAIN_FILE=./toolchain/runtime.cmake -DV2H=ON ..
make -j$(nproc)

The following application file would be generated in the ${PROJECT_PATH}/Q08_object_counter/src/build directory
- object_counter

Application: Deploy Stage

Prerequisites

This section expects the user to have completed Step 7-1 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

microSD card setup is done.

File Configuration

For the ease of deployment all the deployable files and folders are provided in following folders.

Board	`EXE_DIR`
RZ/V2L EVK	exe_v2l
RZ/V2H EVK	exe_v2h

Each folder contains following items.

File	Details
coco/tinyyolov3_onnx	[RZ/V2L only] Model object files for Coco Detection
coco/yolov3_onnx	[RZ/V2H only] Model object files for Coco Detection
coco/coco_class.txt	Label list for Coco Detection
coco/config.ini	User input model config object
animal/animal_onnx	Model object files for Animal Detection
animal/animal_class.txt	Label list for Animal Detection
animal/config.ini	User input model config object
vehicle/vehicle_onnx	Model object files for Vehicle Detection
vehicle/vehicle_class.txt	Label list for Vehicle Detection
vehicle/config.ini	User input model config object
app_conf.ini	User input application config object
object_counter	Application file

Instruction

[For RZ/V2H only] Run following commands to download the necessary file.
Replace each variable according to your board.

cd <path_to_data_folder_on_host>/data/Q08_object_counter/<EXE_PATH> 
wget <URL>

Target	`EXE_PATH`	`URL`	`SO_FILE`	`File Location`
Animal	exe_v2h/animal/animal_onnx	`https://github.com/renesas-rz/rzv_ai_sdk/releases/download/v5.00/Q08_object_counter_animal_deploy_tvm_v2h-v230.so`	`Q08_object_counter_animal_deploy_tvm_v2h-v230.so`	Release v5.00
Vehicle	exe_v2h/vehicle/vehicle_onnx	`https://github.com/renesas-rz/rzv_ai_sdk/releases/download/v5.00/Q08_object_counter_vehicle_deploy_tvm_v2h-v230.so`	`Q08_object_counter_vehicle_deploy_tvm_v2h-v230.so`	Release v5.00
COCO	exe_v2h/coco/yolov3_onnx	`https://github.com/renesas-rz/rzv_ai_sdk/releases/download/v5.00/Q08_object_counter_coco_deploy_tvm_v2h-v230.so`	`Q08_object_counter_coco_deploy_tvm_v2h-v230.so`	Release v5.00

E.g., for Animal counting, use following commands.

cd <path_to_data_folder_on_host>/data/rzv_ai_sdk/Q08_object_counter/exe_v2h/animal/animal_onnx
wget https://github.com/renesas-rz/rzv_ai_sdk/releases/download/v5.00/Q08_object_counter_animal_deploy_tvm_v2h-v230.so

[For RZ/V2H only] Rename the Q08_object_counter_*.so to deploy.so.
```
mv Q08_object_counter_*.so deploy.so
```
Copy the following files to the /home/root/tvm directory of the rootfs (SD Card) for the board.

File Details

All files in EXE_DIR directory Including deploy.so file.

object_counter application file Generated the file according to Application File Generation
Check if libtvm_runtime.so exists under /usr/lib64 directory of the rootfs (SD card) on the board.

File	Details
All files in `EXE_DIR` directory	Including `deploy.so` file.
`object_counter` application file	Generated the file according to Application File Generation

Folder structure in the rootfs (SD Card) would look like:

|-- usr
|   `-- lib64
|       `-- libtvm_runtime.so
`-- home
    `-- root
        `-- tvm
            |-- coco
            |   |-- tinyyolov3_onnx       #RZ/V2L only
            |   |   |-- deploy.json       #RZ/V2L only
            |   |   |-- deploy.params     #RZ/V2L only
            |   |   `-- deploy.so         #RZ/V2L only
            |   |
            |   |-- yolov3_onnx           #RZ/V2H only
            |   |   |-- deploy.json       #RZ/V2H only
            |   |   |-- deploy.params     #RZ/V2H only
            |   |   `-- deploy.so         #RZ/V2H only
            |   |-- coco_class.txt 
            |   `-- config.ini
            |-- animal
            |   |-- animal_onnx
            |   |   |-- deploy.json
            |   |   |-- deploy.params
            |   |   `-- deploy.so
            |   |-- animal_class.txt
            |   `-- config.ini
            |-- vehicle
            |   |-- vehicle_onnx
            |   |   |-- deploy.json
            |   |   |-- deploy.params
            |   |   `-- deploy.so
            |   |-- vehicle_class.txt
            |   `-- config.ini
            |-- app_conf.ini
            `-- object_counter

Note: The directory name could be anything instead of tvm. If you copy the whole EXE_DIR folder on the board, you are not required to rename it tvm.

Application: Run Stage

Prerequisites

This section expects the user to have completed Step 7-3 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

The board setup is done.
The board is booted with microSD card, which contains the application file.

Instruction

On Board terminal, go to the tvm directory of the rootfs.
```
cd /home/root/tvm/
```
Change the values in app_conf.ini as per the requirements. Detailed explanation of the app_conf.ini file is given at below section.
```
vi app_conf.ini
```
Run the application.
```
./object_counter <mode> <camera>
```
- mode options
  
  Value Description
  
  COCO Detects coco objects listed
  
  animal Detects animals listed
  
  vehicle Detects automobiles listed
Note: The mode will be the section name in app_conf.ini file.
- camera options
  
  Value Description
  
  MIPI MIPI camera as input [RZ/V2L only]
  
  USB USB camera as input
For example, to run in "animal" mode with a USB camera, write the following command.
```
./object_counter animal USB
```
Following window shows up on HDMI screen.

RZ/V2L EVK (Animal) RZ/V2H EVK (Animal)

On application window, following information is displayed.
- Camera capture
- AI result
- Processing time
To terminate the application, switch the application window to the terminal by using Super(windows key)+Tab and press ENTER key on the terminal of the board.

Value	Description
COCO	Detects coco objects listed
animal	Detects animals listed
vehicle	Detects automobiles listed

Value	Description
MIPI	MIPI camera as input [RZ/V2L only]
USB	USB camera as input

Application: Configuration

AI Model

RZ/V2L
- Tiny YOLOv3: Darknet
  Dataset: COCO Input size: 1x3x416x416
  Output1 size: 1x13x13x255 (COCO) / 1x13x13x54 (Animal) / 1x13x13x45 (Vehicle)
  Output2 size: 1x26x26x255 (COCO) / 1x26x26x54 (Animal) / 1x26x26x45 (Vehicle)
RZ/V2H
- YOLOv3: Darknet
  Dataset: COCO
  Input size: 1x3x416x416
  Output1 size: 1x13x13x255 (COCO) / 1x13x13x54 (Animal) / 1x13x13x45 (Vehicle)
  Output2 size: 1x26x26x255 (COCO) / 1x26x26x54 (Animal) / 1x26x26x45 (Vehicle)
  Output2 size: 1x52x52x255 (COCO) / 1x52x52x54 (Animal) / 1x52x52x45 (Vehicle)

Dataset

Model	Dataset	Description
coco	Dataset Link	Dataset used is the same as mentioned in the research paper
animal	Dataset Link	Dataset of wildlife in the mixed coniferous broad-leaved forest
vehicle	Site	Combined multiple sources for different classes from the given site. Sources used are listed in below table

Class	Dataset for RZ/V2L EVK	Dataset for RZ/V2H EVK
motorcycle	Dataset	Dataset
bus	Dataset 1, Dataset 2, Dataset 3	Dataset 1, Dataset 2, Dataset 3
car	Dataset 1, Dataset 2	Dataset 1, Dataset 2
policecar	Dataset 1, Dataset 2	Dataset 1, Dataset 2
ambulance	Dataset 1, Dataset 2	Dataset 1, Dataset 2
truck	Dataset 1, Dataset 2	Dataset 1, Dataset 2
bicycle	Dataset 1, Dataset 2	Dataset 1, Dataset 2, Dataset 3
bike	Dataset 1, Dataset 2	Dataset 1, Dataset 2
Auto	Dataset 1, Dataset 2	Dataset 1, Dataset 2
LCV	Dataset 1, Dataset 2	Dataset 1, Dataset 2
Fire engine	Dataset 1, Dataset 2	Dataset 1, Dataset 2

Note: Link for motorcycle dateset has additional classes bus, car and truck which is also used for training

AI inference time

Board	AI model	AI inference time
RZ/V2L EVK	Tiny YOLOv3	Approximately 57 ms
RZ/V2H EVK	YOLOv3	Approximately 26 ms

Processing

Processing	RZ/V2L EVK	RZ/V2H EVK
Pre-processing	Processed by CPU.	Processed by CPU.
Inference	Processed by DRP-AI and CPU.	Processed by DRP-AI and CPU.
Post-processing	Processed by CPU.	Processed by CPU.

Image buffer size

Board	Camera capture buffer size	HDMI output buffer size
RZ/V2L EVK	VGA (640x480) in YUYV format	HD (1280x720) in BGRA format
RZ/V2H EVK	VGA (640x480) in YUYV format	FHD (1920x1080) in BGRA format

Explanation of the `app_conf.ini` file

The section name can be of your choice. And it will be the mode name.
The section should contains three variables - model_path, label_path & config_path.
- The model_path value is the path to the folder containing compiled model. The folder should also contains also contain preprocess folder.
- The label_path value is the path to the label list the model supports.
- The config_path value is the path to the model configuration ini file. Detailed explanation of the config.ini file is given at below section.

Explanation of the `config.ini` file

The [detect] section contains three variables - conf, anchors & objects.
- The conf value is the confidence threshold used for object detection,
- The anchors are the yolo anchors for the object detection.
- The objects represents class to be identified and it can be changed to other classes present on the class label list.
To modify the configuration settings, edit the values in this file using VI Editor.

vi config.ini

Reference

For RZ/V2H EVK, this application supports USB camera only with 640x480 resolution.
FHD resolution is supported by e-CAM22_CURZH camera (MIPI).
Please refer to following URL for how to change camera input to MIPI camera.
https://renesas-rz.github.io/rzv_ai_sdk/latest/about-applications.

License

Apache License 2.0
For third party OSS library, please see the source code file itself.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Q08_object_counter

Q08_object_counter

README.md

Object Counter Application

Application: Overview

Use Cases

Key Features

Supported Product

Demo

Application: Requirements

Hardware Requirements

Application: Build Stage

Prerequisites

Application File Generation

Application: Deploy Stage

Prerequisites

File Configuration

Instruction

Application: Run Stage

Prerequisites

Instruction

Application: Configuration

AI Model

Dataset

AI inference time

Processing

Image buffer size

Explanation of the `app_conf.ini` file

Explanation of the `config.ini` file

Reference

License

Files

Q08_object_counter

Directory actions

More options

Directory actions

More options

Latest commit

History

Q08_object_counter

Folders and files

parent directory

README.md

Object Counter Application

Application: Overview

Use Cases

Key Features

Supported Product

Demo

Application: Requirements

Hardware Requirements

Application: Build Stage

Prerequisites

Application File Generation

Application: Deploy Stage

Prerequisites

File Configuration

Instruction

Application: Run Stage

Prerequisites

Instruction

Application: Configuration

AI Model

Dataset

AI inference time

Processing

Image buffer size

Explanation of the app_conf.ini file

Explanation of the config.ini file

Reference

License

Explanation of the `app_conf.ini` file

Explanation of the `config.ini` file