Egocentric Action Anticipation Based on Unsupervised Gaze Estimation

作     者：ZHONG Cengsi FANG Zhijun GAO Yongbin HUANG Bo ZHONG Cengsi;FANG Zhijun;GAO Yongbin;HUANG Bo

作者机构：School of Electronic and Electrical EngineeringShanghai University of Engineering ScienceShanghai 201620China 

出 版 物：《Wuhan University Journal of Natural Sciences》 (武汉大学学报（自然科学英文版）)

年 卷 期：2021年第26卷第3期

页      面：207-214页

核心收录：

学科分类：08[工学] 080203[工学-机械设计及理论] 0802[工学-机械工程] 

基　　金：Supported by the National Natural Science Foundation of China(61772328) 

主　　题：egocentric video action anticipation gaze estimation 

摘      要：Gaze information is important for finding region of interest(ROI)which implies where the next action will *** gaze estimation does not work on EPIC-Kitchens for lack of ground *** this paper,we develop an unsupervised gaze estimation method that helps with egocentric action *** adopt gaze map as a feature representation,and input it into a multiple modality network jointly with red-green-blue(RGB),optical flow and object *** explore the method on EGTEA *** estimated gaze map is further optimized with dilation and Gaussian filter,masked onto the original RGB frame and encoded as the important gaze *** results outperform the strong baseline Rolling-Unrolling LSTMs(RULSTM),with top-5 accuracy achieving 34.31%on the seen test set(S1)and 22.07%on unseen test set(S2).The accuracy is improved by 0.58%and 0.87%,respectively.