Vision System

Each vision will publish its vision devices frames which were captured. Each vision device should be publishing on a separate topic so that frames don't interfere with one another.

The pipeline for the vision system is as follows:

• Get input image -> rectify image -> Transform to top view -> stitch together front/back view -> detect road features -> lookup location on map -> add absolute measurement to kf
• Get input image -> rectify image -> Transform to front/back view -> detect signs -> lookup location on map -> add absolute measurement to kf
• Get input image -> rectify image -> calculate visual odom -> add relative motion measurement to kf

The following is mostly for detecting obstacles on the road such as another buggy.

• Get input image -> rectify image -> find interesting points -> compare location to last seen location -> build a point cloud

#We use a variant of the bag of words algorithm to classify each part of an image.

Training descriptors of super pixels

Training data is recorded by showing a super pixel and asking them to label the super pixel from a list of labels. Every time that the set of labels is changed or how super pixels are calculated is changed in a major way then all training will have to be redone.

If the descriptor is changed then the training data will not need to be redone. This is because the data needed to recreate a super pixels descriptor is in the Jason file. The trained data will be stored as a jason file that can be loaded at runtime