Mesh-based video objects tracking combining motion and luminance discontinuities criteria

Abstract

A new content based mesh design algorithm for video objects segmentation and tracking is proposed. The algorithm moves the mesh nodes along motion or luminance discontinuities in order to fit the objects boundaries. The motion inside adjacent triangular cells is estimated by competitive forward-backward matching. This approach allows the displacements of the nodes without the remeshing of occlusion regions : uncovered background nor background to be covered. The mesh is deformed under the minimization of an objective function. This function includes a motion energy, a spatial boundary energy and a temporal regularization energy. The motion energy is effective in textured regions while the boundary energy mainly operates in homogeneous one according to a boundary distance map. The temporal regularization energy keeps the objects tracking stable along the sequence due to its object-based approach. Experimental results with both synthetic and real world sequences prove the efficiency of the proposed approach and the cooperative action of the three energy terms.