Rectangling irregular videos by optimal spatio-temporal warping

作     者：Jin-Liang Wu Jun-Jie Shi Lei Zhang Jin-Liang Wu;Jun-Jie Shi;Lei Zhang

作者机构：The 54th Research Institute of CETCShijiazhuang 050050China School of Computer ScienceBeijing Institute of TechnologyBeijing 100081China. 

出 版 物：《Computational Visual Media》 (计算可视媒体（英文版）)

年 卷 期：2022年第8卷第1期

页      面：93-103页

核心收录：

学科分类：08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 

基　　金：supported by the National Natural Science Foundation of China under Grant Nos.61922014 and 61772069 

主　　题：rectangling warping content-preserving 

摘      要：Image and video processing based on geometric principles typically changes the rectangular shape of video frames to an irregular shape. This paper presents a warping based approach for rectangling such irregular frame boundaries in space and time, i.e., making them rectangular again. To reduce geometric distortion in the rectangling process, we employ contentpreserving deformation of a mesh grid with line structures as constraints to warp the frames. To conform to the original inter-frame motion, we keep feature trajectory distribution as constraints during motion compensation to ensure stability after warping the frames. Such spatially and temporally optimized warps enable the output of regular rectangular boundaries for the video frames with low geometric distortion and jitter. Our experiments demonstrate that our approach can generate plausible video rectangling results in a variety of applications.