TITLE: Joint Tracking and Segmentation of Multiple Targets
AUTHOR: Milan, Anton and Leal-Taixe, Laura and Schindler, Konrad and Reid, Ian
FROM: CVPR2015
CONTRIBUTIONS
- A new CRF model taking advantage of both high-level detector responses and low-level superpixel information
- Fully automated segmentation and tracking of an unknown number of targets.
- A complete state representation at every time step could handle occlusions
METHOD
- Generate an overcomplete set of trajectory hypotheses.
- Solve data association problem by optimizing an objective function, which is a multi-label conditional random field (CRF).
SOME DETAILS
The goal is to find the most probable labeling for all nodes given the observations, which is equivalent to
$$ v^{*} = \mathrm{argmin_{v}}E(\nu) $$
in which
$$ E(\nu) = \sum_{s\in\large{\nu}{S}}\phi^{\large{\nu}{S}}(s) + \sum_{d\in\large{\nu}{D}}\phi^{\large{\nu}{D}}(d) + \sum_{(v,w)\in\Large{\varepsilon}}\psi(v,w)+\psi^{\lambda}$$
where \(\phi^{\large{\nu}{S}}\) and \(\phi^{\large{\nu}{D}}\) are unary potential functions for superpixel and detection nodes, respectively, measuring the cost of one detection node in \(\large{\nu}{D}\) or one superpixel node in \(\large{\nu}{S}\) belonging to a certain target; \(\psi(v,w)\) is pairwise edges among superpixels and detections, including spacial and temporal information among superpixels and information among superpixels and detections in the same frame; \(\psi^{\lambda}\) is trajectory cost, containing several constrains of height, shape, dynamics, persistence, image likelihood and parsimony.
ADVANTAGES
- Taking pixel (superpixel) level information in addition to detection results into consideration could handle partial occlusions, which would lead to higher recall.
- Segments could provide considerable information even no reliable detection result exists.
- Modeling multi-targets tracking problem to graph model could take advantage of existing optimization algorithms.
DISADVANTAGES
- Solving CRF problem is slow, needing 12 seconds per frame.
- Can not handle ID switch in two adjacent temporal slidewindows.
OTHER
- Tracking-by-detection has proven to be the most successful strategy to address multi-target tracking problem.
- Noise and imprecise measurements, long-term occlusions, complicated dynamics and target interactions all contributes to the problem’s complexity.