Paper ID | SS-3DPU.8 | ||
Paper Title | A Deep Learning Method for Frame Selection in Videos for Structure from Motion Pipelines | ||
Authors | Francesco Banterle, ISTI-CNR, Italy; Rui Gong, ETH Zurich, Switzerland; Massimiliano Corsini, Fabio Ganovelli, ISTI-CNR, Italy; Luc Van Gool, ETH Zurich, Switzerland; Paolo Cignoni, ISTI-CNR, Italy | ||
Session | SS-3DPU: Special Session: 3D Visual Perception and Understanding | ||
Location | Area B | ||
Session Time: | Tuesday, 21 September, 15:30 - 17:00 | ||
Presentation Time: | Tuesday, 21 September, 15:30 - 17:00 | ||
Presentation | Poster | ||
Topic | Applications of Machine Learning: Machine Learning for 3D Image and Video Processing | ||
IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
Abstract | Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results. |