Paper ID | SS-MMSDF-1.3 | ||
Paper Title | FSFT-NET: FACE TRANSFER VIDEO GENERATION WITH FEW-SHOT VIEWS | ||
Authors | Luchuan Song, Guojun Yin, Bin Liu, Unversity of Science and Technology of China, China; Yuhui Zhang, University of Science and Technology of China, China; Nenghai Yu, Unversity of Science and Technology of China, China | ||
Session | SS-MMSDF-1: Special Session: AI for Multimedia Security and Deepfake 1 | ||
Location | Area B | ||
Session Time: | Monday, 20 September, 15:30 - 17:00 | ||
Presentation Time: | Monday, 20 September, 15:30 - 17:00 | ||
Presentation | Poster | ||
Topic | Special Sessions: Artificial Intelligence for Multimedia Security and Deepfake | ||
IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
Abstract | To transfer head pose and expression with few photographs is a novel yet challenging task in deepfake generation. Despite impressive results have been achieved in related works, there are still two limitations in the existing methods: 1) most of the methods are based on computer graphics, which take a lot of computing resources,while lacking of generalization for different identity, 2) few-shot based methods cannot handle the few-shot style transfer video generation. To address these distortion problems, we propose a novel deep learning framework, named as Few Shot Face Transfer Networks(FSFT-Net) which works for the face transfer video generation. The proposed FSFT-Net driven by arbitrary portrait video involves a cascaded-based style generator to synthesize stable video with few free-view images. In addition, the frame and video discriminators are adopted for optimization of the proposed generator. The FSFT-Net performs long-term adversarial training on large-scale video datasets. Extensive experiments demonstrate that our FSFT-Net outperforms state-of-the-art methods both quantitatively and qualitatively results. |