Paper ID | ARS-8.1 | ||
Paper Title | Progressive Knowledge Distillation for Early Action Recognition | ||
Authors | Vinh Tran, Niranjan Balasubramanian, Minh Hoai Nguyen, Stony Brook University, United States | ||
Session | ARS-8: Image and Video Mid-Level Analysis | ||
Location | Area I | ||
Session Time: | Monday, 20 September, 13:30 - 15:00 | ||
Presentation Time: | Monday, 20 September, 13:30 - 15:00 | ||
Presentation | Poster | ||
Topic | Image and Video Analysis, Synthesis, and Retrieval: Image & Video Mid-Level Analysis | ||
IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
Abstract | We present a novel framework to train a recurrent neural network for early recognition of human action, which is an important but challenging task given the need to recognize an on-going action based on partial observation. Our framework is based on knowledge distillation, where the network for early recognition is a student model, and it is trained by distilling the knowledge from a teacher model that has superior knowledge by peeking into the future and incorporating extra observations about the action in consideration. This framework can be used in both supervised and semi-supervised learning settings, being able to utilize both the labeled and unlabeled training data. Experiments on the UCF101, SYSU 3DHOI, and NTU RGB-D datasets show the effectiveness of knowledge distillation for early recognition, including situations where we only have a small amount of annotated training data. |