Detecting human-object interaction with multi-level pairwise feature network

作     者：Hanchao Liu Tai-Jiang Mu Xiaolei Huan 

作者机构：Key Laboratory of Pervasive ComputingMinistry of EducationBNRistDepartment of Computer Science and TechnologyTsinghua UniversityBeijing 100084China College of Information Sciences and TechnologyPennsylvania State UniversityUniversity ParkPA 16802USA. 

出 版 物：《Computational Visual Media》 (计算可视媒体（英文版）)

年 卷 期：2021年第7卷第2期

页      面：229-239页

核心收录：

学科分类：08[工学] 080203[工学-机械设计及理论] 0835[工学-软件工程] 0802[工学-机械工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：supported by the National Natural Science Foundation of China(Project No.61902210) a Research Grant of Beijing Higher Institution Engineering Research Center and the Tsinghua–Tencent Joint Laboratory for Internet Innovation Technology 

主　　题：human–object interaction detection pairwise feature network deep learning multi-level object instance 

摘      要：Human–object interaction(HOI)detection is crucial for human-centric image understanding which aims to infer human,action,object triplets within an *** studies often exploit visual features and the spatial configuration of a human–object pair in order to learn the action linking the human and object in the *** argue that such a paradigm of pairwise feature extraction and action inference can be applied not only at the whole human and object instance level,but also at the part level at which a body part interacts with an object,and at the semantic level by considering the semantic label of an object along with human appearance and human–object spatial configuration,to infer the *** thus propose a multi-level pairwise feature network(PFNet)for detecting human–object *** network consists of three parallel streams to characterize HOI utilizing pairwise features at the above three levels;the three streams are finally fused to give the action *** experiments show that our proposed PFNet outperforms other state-of-the-art methods on the VCOCO dataset and achieves comparable results to the state-of-the-art on the HICO-DET dataset.