ov_core::TrackBase class

Visual feature tracking base class.

This is the base class for all our visual trackers. The goal here is to provide a common interface so all underlying trackers can simply hide away all the complexities. We have something called the "feature database" which has all the tracking information inside of it. The user can ask this database for features which can then be used in an MSCKF or batch-based setting. The feature tracks store both the raw (distorted) and undistorted/normalized values. Right now we just support two camera models, see: undistort_point_brown() and undistort_point_fisheye().

This base class also handles most of the heavy lifting with the visualization, but the sub-classes can override this and do their own logic if they want (i.e. the TrackAruco has its own logic for visualization). This visualization needs access to the prior images and their tracks, thus must synchronise in the case of multi-threading. This shouldn't impact performance, but high frequency visualization calls can negatively effect the performance.

Derived classes

class TrackAruco
Tracking of OpenCV Aruoc tags.
class TrackDescriptor
Descriptor-based visual tracking.
class TrackKLT
KLT tracking of features.
class TrackSIM
Simulated tracker for when we already have uv measurements!

Public types

enum HistogramMethod { NONE, HISTOGRAM, CLAHE }
Desired pre-processing image method.

Constructors, destructors, conversion operators

TrackBase(std::unordered_map<size_t, std::shared_ptr<CamBase>> cameras, int numfeats, int numaruco, bool stereo, HistogramMethod histmethod)
Public constructor with configuration variables.

Public functions

void feed_new_camera(const CameraData& message) pure virtual
Process a new image.
void display_active(cv::Mat& img_out, int r1, int g1, int b1, int r2, int g2, int b2) virtual
Shows features extracted in the last image.
void display_history(cv::Mat& img_out, int r1, int g1, int b1, int r2, int g2, int b2, std::vector<size_t> highlighted = {}) virtual
Shows a "trail" for each feature (i.e. its history)
auto get_feature_database() -> std::shared_ptr<FeatureDatabase>
Get the feature database with all the track information.
void change_feat_id(size_t id_old, size_t id_new)
Changes the ID of an actively tracked feature to another one.

Protected variables

std::unordered_map<size_t, std::shared_ptr<CamBase>> camera_calib
Camera object which has all calibration in it.
std::shared_ptr<FeatureDatabase> database
Database with all our current features.
std::map<size_t, bool> camera_fisheye
If we are a fisheye model or not.
int num_features
Number of features we should try to track frame to frame.
bool use_stereo
If we should use binocular tracking or stereo tracking for multi-camera.
HistogramMethod histogram_method
What histogram equalization method we should pre-process images with?
std::vector<std::mutex> mtx_feeds
Mutexs for our last set of image storage (img_last, pts_last, and ids_last)
std::map<size_t, cv::Mat> img_last
Last set of images (use map so all trackers render in the same order)
std::map<size_t, cv::Mat> img_mask_last
Last set of images (use map so all trackers render in the same order)
std::unordered_map<size_t, std::vector<cv::KeyPoint>> pts_last
Last set of tracked points.
std::unordered_map<size_t, std::vector<size_t>> ids_last
Set of IDs of each current feature in the database.
std::atomic<size_t> currid
Master ID for this tracker (atomic to allow for multi-threading)

Function documentation

ov_core::TrackBase::TrackBase(std::unordered_map<size_t, std::shared_ptr<CamBase>> cameras, int numfeats, int numaruco, bool stereo, HistogramMethod histmethod)

Public constructor with configuration variables.

Parameters
cameras camera calibration object which has all camera intrinsics in it
numfeats number of features we want want to track (i.e. track 200 points from frame to frame)
numaruco the max id of the arucotags, so we ensure that we start our non-auroc features above this value
stereo if we should do stereo feature tracking or binocular
histmethod what type of histogram pre-processing should be done (histogram eq?)

void ov_core::TrackBase::feed_new_camera(const CameraData& message) pure virtual

Process a new image.

Parameters
message Contains our timestamp, images, and camera ids

void ov_core::TrackBase::display_active(cv::Mat& img_out, int r1, int g1, int b1, int r2, int g2, int b2) virtual

Shows features extracted in the last image.

Parameters
img_out image to which we will overlayed features on
r1 first color to draw in
g1 first color to draw in
b1 first color to draw in
r2 second color to draw in
g2 second color to draw in
b2 second color to draw in

void ov_core::TrackBase::display_history(cv::Mat& img_out, int r1, int g1, int b1, int r2, int g2, int b2, std::vector<size_t> highlighted = {}) virtual

Shows a "trail" for each feature (i.e. its history)

Parameters
img_out image to which we will overlayed features on
r1 first color to draw in
g1 first color to draw in
b1 first color to draw in
r2 second color to draw in
g2 second color to draw in
b2 second color to draw in
highlighted unique ids which we wish to highlight (e.g. slam feats)

std::shared_ptr<FeatureDatabase> ov_core::TrackBase::get_feature_database()

Get the feature database with all the track information.

Returns FeatureDatabase pointer that one can query for features

void ov_core::TrackBase::change_feat_id(size_t id_old, size_t id_new)

Changes the ID of an actively tracked feature to another one.

Parameters
id_old Old id we want to change
id_new Id we want to change the old id to

This function can be helpfull if you detect a loop-closure with an old frame. One could then change the id of an active feature to match the old feature id!