Content-driven Video Retargeting

Lior Wolf Moshe Guttmann Daniel Cohen-Or

International Conf. on Computer Vision, ICCV 2007

Abstract:

Video retargeting is the process of transforming an existing video to fit the dimensions of an arbitrary display.
A compelling retargeting aims at preserving the viewers' experience by maintaining the information content of important regions in the
frame, whilst keeping their aspect ratio.

An efficient  algorithm for video retargeting is introduced. It consists of two stages. First, the frame is analyzed to detect the importance
of each region in the frame.  Then, a transformation that respects the analysis shrinks less important regions more than important ones.
Our analysis is fully automatic and based on local saliency, motion detection and object detectors.
The performance of the proposed algorithm is demonstrated on a variety of video sequences, and compared to the state of the art in image retargeting.

System overview:

A saliency score is computed for each frame. Next, and optimization stage recovers the retargeting warp. Finally, the warp is applied to the original frame.        

Video:

Windows movie format, ~30Mb

Comparison with other recent systems:

Recently, two systems introduce photorealistic solutions for content-aware remapping of still images. Gal, Sorkine and Cohen-Or introduce (Feature-aware texturing, Eurographics Symposium on Rendering 2006) a method to modify an arbitrarily image warp while preserving the shape of important regions by constraining their deformation to be a similarity or a rigidity transformation. In this system, which is designed for still images, the important regions are manually marked by the user.

Avidan and Shamir (Seam Carving for Content-Aware Image Resizing, SIGGRAPH 2007, marked below as [1]) propose an impressive system, where the retargeting is applied by reducing the width (or the height) of the image by one pixel at a time, through deleting a vertical (or horizontal) connected paths of low importance pixels. While they show excellent results on images, their method seems to be overly expensive to be extended to video and their solution is greedy and discrete (see some comparison below). Unlike the above two, our work is designed for video, and in particular for video streaming.

BibTeX entry:

@inproceedings{WOLFGCO2007,
author = {Lior Wolf and Moshe Guttmann and Daniel Cohen-Or},
title = {Non-homogeneous Content-driven Video-retargeting},
booktitle = { Proceedings of the Eleventh IEEE International Conference on Computer Vision (ICCV-07)},
year = {2007},
}

Contact the authors