TrackKLT class
KLT tracking of features.
Contents
This is the implementation of a KLT visual frontend for tracking sparse features. We can track either monocular cameras across time (temporally) along with stereo cameras which we also track across time (temporally) but track from left to right to find the stereo correspondence information also. This uses the calcOpticalFlowPyrLK OpenCV function to do the KLT tracking.
Base classes
- class TrackBase
- Visual feature tracking base class.
Constructors, destructors, conversion operators
- TrackKLT(std::unordered_map<size_t, std::shared_ptr<CamBase>> cameras, int numfeats, int numaruco, bool stereo, HistogramMethod histmethod, int fast_threshold, int gridx, int gridy, int minpxdist) explicit
- Public constructor with configuration variables.
Public functions
- void feed_new_camera(const CameraData& message) override
- Process a new image.
Protected functions
- void feed_monocular(const CameraData& message, size_t msg_id)
- Process a new monocular image.
- void feed_stereo(const CameraData& message, size_t msg_id_left, size_t msg_id_right)
- Process new stereo pair of images.
- void perform_detection_monocular(const std::vector<cv::Mat>& img0pyr, const cv::Mat& mask0, std::vector<cv::KeyPoint>& pts0, std::vector<size_t>& ids0)
- Detects new features in the current image.
- void perform_detection_stereo(const std::vector<cv::Mat>& img0pyr, const std::vector<cv::Mat>& img1pyr, const cv::Mat& mask0, const cv::Mat& mask1, size_t cam_id_left, size_t cam_id_right, std::vector<cv::KeyPoint>& pts0, std::vector<cv::KeyPoint>& pts1, std::vector<size_t>& ids0, std::vector<size_t>& ids1)
- Detects new features in the current stereo pair.
- void perform_matching(const std::vector<cv::Mat>& img0pyr, const std::vector<cv::Mat>& img1pyr, std::vector<cv::KeyPoint>& pts0, std::vector<cv::KeyPoint>& pts1, size_t id0, size_t id1, std::vector<uchar>& mask_out)
- KLT track between two images, and do RANSAC afterwards.
Function documentation
ov_core::
Public constructor with configuration variables.
| Parameters | |
|---|---|
| cameras | camera calibration object which has all camera intrinsics in it |
| numfeats | number of features we want want to track (i.e. track 200 points from frame to frame) |
| numaruco | the max id of the arucotags, so we ensure that we start our non-auroc features above this value |
| stereo | if we should do stereo feature tracking or binocular |
| histmethod | what type of histogram pre-processing should be done (histogram eq?) |
| fast_threshold | FAST detection threshold |
| gridx | size of grid in the x-direction / u-direction |
| gridy | size of grid in the y-direction / v-direction |
| minpxdist | features need to be at least this number pixels away from each other |
void ov_core::
Process a new image.
| Parameters | |
|---|---|
| message | Contains our timestamp, images, and camera ids |
void ov_core::
Process a new monocular image.
| Parameters | |
|---|---|
| message | Contains our timestamp, images, and camera ids |
| msg_id | the camera index in message data vector |
void ov_core::
Process new stereo pair of images.
| Parameters | |
|---|---|
| message | Contains our timestamp, images, and camera ids |
| msg_id_left | first image index in message data vector |
| msg_id_right | second image index in message data vector |
void ov_core::
Detects new features in the current image.
| Parameters | |
|---|---|
| img0pyr | image we will detect features on (first level of pyramid) |
| mask0 | mask which has what ROI we do not want features in |
| pts0 | vector of currently extracted keypoints in this image |
| ids0 | vector of feature ids for each currently extracted keypoint |
Given an image and its currently extracted features, this will try to add new features if needed. Will try to always have the "max_features" being tracked through KLT at each timestep. Passed images should already be grayscaled.
void ov_core::
Detects new features in the current stereo pair.
| Parameters | |
|---|---|
| img0pyr | left image we will detect features on (first level of pyramid) |
| img1pyr | right image we will detect features on (first level of pyramid) |
| mask0 | mask which has what ROI we do not want features in |
| mask1 | mask which has what ROI we do not want features in |
| cam_id_left | first camera sensor id |
| cam_id_right | second camera sensor id |
| pts0 | left vector of currently extracted keypoints |
| pts1 | right vector of currently extracted keypoints |
| ids0 | left vector of feature ids for each currently extracted keypoint |
| ids1 | right vector of feature ids for each currently extracted keypoint |
This does the same logic as the perform_
void ov_core::
KLT track between two images, and do RANSAC afterwards.
| Parameters | |
|---|---|
| img0pyr | starting image pyramid |
| img1pyr | image pyramid we want to track too |
| pts0 | starting points |
| pts1 | points we have tracked |
| id0 | id of the first camera |
| id1 | id of the second camera |
| mask_out | what points had valid tracks |
This will track features from the first image into the second image. The two point vectors will be of equal size, but the mask_out variable will specify which points are good or bad. If the second vector is non-empty, it will be used as an initial guess of where the keypoints are in the second image.