SteadyFlow: Spatially Smooth Optical Flow for Video StabilizationShuaicheng Liu1 Lu yuan2 Ping Tan1 Jian Sun2 1. National University of Singapore 2. Microsoft Research |
---|
Abstract: |
---|
We propose a novel motion model, SteadyFlow, to represent the motion between neighboring video frames for stabilization. A SteadyFlow is a specific optical flow by enforcing strong spatial coherence, such that smoothing feature trajectories can be replaced by smoothing pixel profiles, which are motion vectors collected at the same pixel location in the SteadyFlow over time. In this way, we can avoid brittle feature tracking in a video stabilization system. Besides, SteadyFlow is a more general 2D motion model which can deal with spatially-variant motion. We initialize the SteadyFlow by optical flow and then discard discontinuous motions by a spatial-temporal analysis and fill in missing regions by motion completion. Our experiments demonstrate the effectiveness of our stabilization on real-world challenging videos.
|
Paper [PDF]Related Projects Shuaicheng Liu, Mingyu Li, Shuyuan Zhu, Bing Zeng: CodingFlow: Enable Video Coding for Video Stabilization. IEEE Transactions on Image Processing (TIP), vol. 26, no. 7, pp. 3291-3302, 2017. [PDF] Shuaicheng Liu, Ping Tan, Lu Yuan, Jian Sun, Bing Zeng: MeshFlow: Minimum Laency Online Video Stabilization. European Conference on Computer Vision (ECCV). 2016. [PDF][Video][Model Code] Shuaicheng Liu, Lu yuan, Ping Tan, Jian Sun. Bundled Camera Paths for Video Stabilization. ACM Transactions on Graphics (Proceeding of SIGGRAPH) 2013. [PDF][project page] Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun: Video Stabilization with a Depth Camera. IEEE Conference on Computer Vision and Patten Recognition(CVPR) 2012 [PDF][project page] |
Video Spotlight |
---|
Full Demo Video: download [64Mb] |
Downloads:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Limitations |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Our spatial-temporal analysis failed to distinguish foreground and background when videos contain dominate foregrounds, (foregrounds occupy more than half area of a frame and exist for a long time).
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|