Skip to content

Latest commit

 

History

History
179 lines (128 loc) · 4.78 KB

README.md

File metadata and controls

179 lines (128 loc) · 4.78 KB
Raw

DOI

Appearance-based methods for visual localisation

This is a MATLAB library to extract visual descriptors and implement a bag-of- visual-words pipeline from video sequences taken by multiple users in order to provide localisation.

The code is customised and ready to be used with the RSM dataset (http://rsm.bicv.org) but can be used on any sort of image sequences if the directory paths are correctly specified.

Current implemented descriptor extraction methods (description below): LW_COLOR, SIFT, DSIFT, SF_GABOR, ST_GABOR, ST_GAUSS

Current supported format of the sequences: jpg

Authors:

Web: http://www.bicv.org

Date: v4.1 11/2015

Requirements:

SIFT, DSIFT, VLAD and kernel implementations require VLFEAT Clustering requires INRIA's Yael K-means

Running Instructions:

Rename initialize.m.template to initialize.m

cp initialize.m.template initialize.m

Run main.m

main

Detailed Instructions:

Parameter selection

  • Dependency paths: include the paths to the dependencies.

A version of these libraries is included in the Downloads section of the repository

  • Parameter selection.

Select your choice from the following parameters in the params structure before continuing:

params = struct(...
    'descriptor',    'ST_GAUSS',...  % SIFT, DSIFT, SF_GABOR, ST_GABOR, ST_GAUSS,
    'corridors',     1:6,... % Corridors to run [1:6] (RSM v6.0)
    'passes',        1:10,... % Passes to run [1:10] (RSM v6.0)
    'trainingSet',   [1:3,5], ... 
    'datasetDir',    '/data/datasets/RSM/visual_paths/v6.0',...   % The root path of the RSM dataset
    'frameDir',      'frames_resized_w208p',... % Folder name where all the frames have been extracted.
    'descrDir',  ...
    '/data/datasets/RSM/descriptors', ...
    'dictionarySize', 400, ...
    'dictPath',       '/data/datasets/RSM/dictionaries', ...
    'encoding', 'HA', ... % 'HA', 'VLAD', 'LLC'
    'kernel', 'chi2', ... % 'chi2', 'Hellinger'
    'kernelPath', '/data/datasets/RSM/kernels', ...
    'metric', 'max', ...
    'groundTruthPath', './ground_truth', ...
    'debug', 1 ... % 1 shows waitbars, 0 does not.
    );

These parameters are the following

  • datasetDir: The root path of the RSM dataset
  • corridors: Corridors to run [1:6] (RSM v6.0)
  • passes: Passes to run [1:10] (RSM v6.0)
  • trainingSet: training set to use for dictionary construction
  • frameDir: Folder name where all the frames have been extracted.
  • descrDir
  • descriptor: Type of descriptors to be calculated. To choose from
    • LW_COLOR: Lightweight spatio-temporal colour descriptor
    • SIFT: keypoint based SIFT descriptors
    • DSIFT: Dense SIFT
    • SF_GABOR: Frame-based DAISY-like descriptors
    • ST_GABOR: Spatio-temporal Gabors.
    • ST_GAUSS: Spatio-temporal, Spatial Derivative, Temporal Gaussian
  • dictionarySize: number of visual words (parameter k in k-means)
  • dictPath: directory where to store the created dictionaries.
  • encoding: encoding method
  • kernel:
  • kernelPath:
  • metric:
  • groundTruthPath:
  • debug:

Descriptor generation

computeDescriptors(params);

Bag of Words pipeline

  • create_dictionaries (k-means vector quantization)
clusterDescriptors

% OR

clusterDescriptorsSparse (for Keypoint-SIFT)
  • BOVW encoding (Hard assigment, VLAD, or LLC)
hovwEncoding

%       Remember to modify the parameters of the encoding, which will automatically call
%       encode_hovw_METHOD/encode_hovw_METHOD_sparse (for Keypoint-SIFT),
%       where METHOD = 
%         HA, "Visual categorization with bags of keypoints", Dance et al., 2004,
%         VLAD, "Aggregating local descriptors into a compact image representation", Jegou et al., 2010,
%         LLC, "Locality-constrained Linear Coding for Image Classification", Wang et al., 2010.
  • Kernels for histograms
runKernelHA

% OR

runKernelHellinger
  • Run evaluation routine to add the error measurement to the kernels.
run_evaluation_nn_VW
  • [Optional: Move kernels to one folder]
#!/bin/bash
cd /folder/to/kernels
mkdir all_chi2
find . -name *chi2*.mat -exec cp -vf {} all_chi2/ \; # if chi2 kernel
mkdir all_Hellinger
find . -name *Hellinger*.mat -exec cp -vf {} all_chi2/ \;
  • Generate PDF results and plots with
results_generation.m