Our system enables creating realistic tracking shots from shot video sequences, with little user input. The resulting tracking shot depicts object motion in a still image by adaptively blurring the background according to both foreground motion and scene structure in 3D.

Panning and tracking shots are popular photography techniques in which the camera tracks a moving object and keeps it at the same position, resulting in an image where the moving foreground is sharp but the background is blurred accordingly, creating an artistic illustration of the foreground motion. Such shots however are hard to capture even for professionals, especially when the foreground motion is complex (e.g., non-linear motion trajectories).

In this work we propose a system to generate realistic, 3D-aware tracking shots from consumer videos. We show how computer vision techniques such as segmentation and structure-from-motion can be used to lower the barrier and help novice users create high quality tracking shots that are physically plausible. We also introduce a pseudo 3D approach for relative depth estimation to avoid expensive 3D reconstruction for improved robustness and a wider application range. We validate our system through extensive quantitative and qualitative evaluations.

Paper: [PDF 4MB]

Video:  Download [TrackCam.mp4 (34MB)]

Data: Download [26MB] (inputs, results, masks)

Acknowledgements

We thank the reviewers for their constructive comments and the user study participants for their time. We give special thanks to Prof. Michael S.Brown for his voice.

BibTex

@article{liu2014trackcam,
title={TrackCam:3D-aware Tracking Shots from Consumer Video},
author={Liu, Shuaicheng and Wang, Jue and Cho, Sunghyun and Tan, Ping},
journal={ACM Transactions on Graphics (TOG)},
volume={33},
number={6},
pages={198},
year={2014},
publisher={ACM}
}