Paper ID | MLR-APPL-IVASR-1.10 | ||
Paper Title | Application-agnostic spatio-temporal hand graph representations for stable activity understanding | ||
Authors | Pratyusha Das, Antonio Ortega, University of Southern California, United States; Siheng Chen, Shanghai Jiao Tong University, China; Hassan Mansour, Anthony Vetro, Mitsubishi Electric Research Labs, United States | ||
Session | MLR-APPL-IVASR-1: Machine learning for image and video analysis, synthesis, and retrieval 1 | ||
Location | Area D | ||
Session Time: | Monday, 20 September, 13:30 - 15:00 | ||
Presentation Time: | Monday, 20 September, 13:30 - 15:00 | ||
Presentation | Poster | ||
Topic | Applications of Machine Learning: Machine learning for image & video analysis, synthesis, and retrieval | ||
IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
Abstract | Understanding complex hand actions, such as assembly tasks or kitchen activities, from hand skeleton data is an important yet challenging task. This paper introduces a hand graph-based spatio-temporal feature extraction method which uniquely represents complex hand action in an unsupervised manner. To evaluate the efficacy of the proposed representation, we consider action segmentation and recognition tasks. The segmentation problem involves an assembling task in an industrial setting, while the recognition problem deals with kitchen and office activities. Additionally, for both segmentation and recognition models, we propose notions of stability, which are used to demonstrate the robustness of our proposed approach. We introduce validation loss stability (ValS) and estimation stability with cross-validation (EtS) to analyze the robustness of any supervised classification model. The proposed method shows comparable classification performance with state of the art methods, but it achieves significantly better accuracy and stability in a cross-person setting. |