A Hybrid Approach for Near-Range Video Stabilization

We present a hybrid approach that combines the benefits of 2D methods with those of 3D methods for near-range video stabilization. Near-range videos contain objects that are close to the camera. These videos often contain discontinuous-depth variation (DDV), which is the main challenge to the existing video stabilization methods. Traditionally, 2D methods are robust to various camera motions (e.g., quick rotation and zooming) under scenes with continuous-depth variation (CDV). However, in presence of DDV, they often generate wobbled results due to the limited ability of their 2D motion models. Alternatively, 3D meth ods are more robust in handling near-range videos. We show that by compensating rotational motions and ignoring translational motions, near-range videos can be successfully stabilized without sacrificing too much stability. However, it is time-consuming to reconstruct the 3D structures for the entire video and sometimes even impossible due to rapid camera motions. In this paper, we aim to combine the advantages of 2D and 3D methods, yielding a hybrid approach that is robust to various camera motions and can handle the near-range scenarios well. In particular, we partition the input video into CDV and DDV segments automatically. Then, the 2D and 3D approaches are adopted for CDV and DDV clips, respectively. Finally, these segments are stitched seamlessly via a constrained optimization. We validate our method on a large variety of consumer videos.

Paper [PDF]

Related Projects

Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun: SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE Conference on Computer Vision and Patten Recognition(CVPR) 2014 [PDF][project page]

Shuaicheng Liu, Lu yuan, Ping Tan, Jian Sun. Bundled Camera Paths for Video Stabilization. ACM Transactions on Graphics (Proceeding of SIGGRAPH) 2013. [PDF][project page]

Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun: Video Stabilization with a Depth Camera. IEEE Conference on Computer Vision and Patten Recognition(CVPR) 2012 [PDF][project page]

Pipeline:

Our system pipeline: the input video (a) is partitioned into CDV segments (b) and DDV segments (c). DDV segments are stabilized by smoothing their camera rotational motion using the 3D stabilization method. The stabilized DDV segments (d) are inserted back to the original video, which generates a partially stabilized video (e). The bundled paths stabilization is applied to the CDV segments contained in (e), where the stabilized DDV segments are encouraged to keep their positions while the CDV segments are adjusted to merge with DDV segments seamlessly to produce the final result (f).