Hidden Two-Stream Collaborative Learning Network for Action Recognition

作     者：Shuren Zhou Le Chen Vijayan Sugumaran 

作者机构：School of Computer and Communication EngineeringChangsha University of Science and TechnologyChangsha410114China Department of Decision and Information SciencesSchool of Business AdministrationOakland UniversityRochester48309USA 

出 版 物：《Computers, Materials & Continua》 (计算机、材料和连续体（英文）)

年 卷 期：2020年第63卷第6期

页      面：1545-1561页

核心收录：

学科分类：0831[工学-生物医学工程（可授工学、理学、医学学位）] 0808[工学-电气工程] 0809[工学-电子科学与技术（可授工学、理学学位）] 08[工学] 0805[工学-材料科学与工程（可授工学、理学学位）] 0701[理学-数学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 0801[工学-力学（可授工学、理学学位）] 

基　　金：This work was supported by the Scientific Research Fund of Hunan Provincial Education Department of China(Project No.17A007) the Teaching Reform and Research Project of Hunan Province of China(Project No.JG1615). 

主　　题：Action recognition collaborative learning optical flow 

摘      要：The two-stream convolutional neural network exhibits excellent performance in the video action recognition.The crux of the matter is to use the frames already clipped by the videos and the optical flow images pre-extracted by the frames,to train a model each,and to finally integrate the outputs of the two models.Nevertheless,the reliance on the pre-extraction of the optical flow impedes the efficiency of action recognition,and the temporal and the spatial streams are just simply fused at the ends,with one stream failing and the other stream succeeding.We propose a novel hidden two-stream collaborative(HTSC)learning network that masks the steps of extracting the optical flow in the network and greatly speeds up the action recognition.Based on the two-stream method,the two-stream collaborative learning model captures the interaction of the temporal and spatial features to greatly enhance the accuracy of recognition.Our proposed method is highly capable of achieving the balance of efficiency and precision on large-scale video action recognition datasets.