Skip to content

Home

Jump to bottom
Charlie edited this page Apr 27, 2016 · 1 revision

Welcome to the PRiMEStereoMatch wiki!

Theoretical Background

A heterogeneous and fully parallel stereo matching algorithm for depth estimation. Stereo Matching is based on the disparity estimation algorithm, an algorithm designed to calculate 3D depth information about a scene from a pair of 2D images captured by a stereoscopic camera. The algorithm contains the following stages:

  • Cost Volume Construction - weighted absolute difference of colours and gradients function.
  • Cost Volume Filtering - local adaptive support weight (ADSW) Guided Image Filter (GIF) function.
  • Disparity Selection - winner-takes-all minimum cost search and corresponding disparity selection.
  • Post Processing - left-right occlusion check, invalid pixel removal and weight-median filtering.

Implementation Details

  • All stages of the algorithm are developed in both C++ and OpenCL.
    • C++ parallelism is introduced via the POSIX threads (pthreads) library. Disparity level parallelism, enabling up to 64 concurrent threads, is supported.
    • OpenCL parallelism is inherent through the concurrent execution of kernels on an OpenCL-compatible device. The optimum level of parallelism will be device-specific.
  • Support for live video disparity estimation using the OpenCV VideoCapture interface as well as standard static image computation.
  • Embedded support for experimentation with the OpenCV standard Semi-Global Block Matching (SGBM) algorithm.

Installation

Prerequisites

  • Hardware:
    • Development Platform - optionally but ideally including OpenCL compatible devices
  • Software:

Compilation

  • Download project folder to your platform.
  • Enter the base directory using cd DE_APP.
  • Natively compile the project using make -j8.
    • Adjust -j8 to suit the number of simultaneous threads supported on your platform.

Deployment

  • Run the application using ./bin/Release/DE_APP <program arguments>

  • The following program arguments must be specified:

    • Matching Algorithm type:
      • STEREO_GIF - Guided Image Filter
      • STEREO_SGBM - Semi Global Block Matching
    • Media type:
      • VIDEO
      • IMAGE

  • When specifying the VIDEO media type, the following optional arguments can be included:

    • RECALIBRATE - recalculate the intrinsic and extrinsic parameters of the stereo camera. Previously captured chessboard images must be supplied if the RECAPTURE flag is not also set.
    • RECAPTURE - record chessboard image pairs in preparation for calibration. A chessboard image must be presented in front of the stereo camera and in full view of both cameras.

  • For example, to run with the guided image filter algorithm using a stereo camera, specify:

    • ./bin/Release/DE_APP STEREO_GIF VIDEO

  • To run with calibration and capture beforehand, specify:

    • ./bin/Release/DE_APP STEREO_GIF VIDEO RECALIBRATE RECAPTURE

  • Image disparity estimation is achieved using for example:

    • ./bin/Release/DE_APP STEREO_GIF IMAGE left_img.png right_img.png

  • The first time the application is deployed using a stereo camera, the RECALIBRATE and RECAPTURE flags must be set in order to capture chessboard image to calculate the intrinsic and extrinsic parameters.

  • This process only needs to be repeated if the relative orientations of the left and right cameras are changed or a different resolution is specified.

  • Once the intrinsic and extrinsic parameters have been calucalted and saved to .yml files, the application can be re-run with the same camera without needing to recalibrate as teh parameters will be loaded from these files. The files can be found in the data directory.

Additional Resources

Directory Structure

DE_APP		- Project top level directory

folders:
	assets			- OpenCL kernel files
	bin				- binary executable files
	common			- OpenCL common utility functions (C) ARM
	data			- program data including input images, stereo camera parameters, calibration images, etc
	include			- Project header files (h/hpp)
	src				- Project source files (c/cpp)
	
files:
	cbp2make.linux-x86_64 	- codeblocks to makefile synthesis tool (for x86_64 PC)
	cbp2make_usage.txt		- cbp2make tool manual
	DE_APP.cbp				- Code::Blocks project file
	DE_APP.depend			- Code::Blocks settings file
	DE_APP.layout			- Code::Blocks settings file
	main.cpp				- main C++ file
	Makefile				- project Makefile
Clone this wiki locally