Vision System

Each vision will publish its vision devices frames which were captured. Each vision device should be publishing on a separate topic so that frames don't interfere with one another.

The pipeline for the vision system is as follows:

• Get input image -> rectify image -> Transform to top view -> stitch together front/back view -> detect road features -> lookup location on map -> add absolute measurement to kf
• Get input image -> rectify image -> Transform to front/back view -> detect signs -> lookup location on map -> add absolute measurement to kf
• Get input image -> rectify image -> calculate visual odom -> add relative motion measurement to kf

The following is mostly for detecting obstacles on the road such as another buggy.

• Get input image -> rectify image -> find interesting points -> compare location to last seen location -> build a point cloud

Training descriptors of super pixels

Training data is recorded by showing a super pixel and asking them to label the super pixel from a list of labels. Every time that the set of labels is changed or how super pixels are calculated is changed in a major way then all training will have to be redone.

The trained data will be stored as a jason file that can be loaded at runtime