ピラミッドグラフ畳み込みネットワークによる人物-物体インタラクションの時空間関係の理解
Understanding Spatio-Temporal Relations in Human-Object Interaction using Pyramid Graph Convolutional Network
人物と物体の関係をグラフで表現し、注意機構付きグラフ畳み込みと時間ピラミッドプーリングを組み合わせて、行動のラベルと時間的構造を同時に認識する手法を提案した論文。
著者: Hao Xing, Darius Burschka
分類: cs.CV, cs.RO
原文アブストラクト
Human activities recognition is an important task for an intelligent robot, especially in the field of human-robot collaboration, it requires not only the label of sub-activities but also the temporal structure of the activity. In order to automatically recognize both the label and the temporal structure in sequence of human-object interaction, we propose a novel Pyramid Graph Convolutional Network (PGCN), which employs a pyramidal encoder-decoder architecture consisting of an attention based graph convolution network and a temporal pyramid pooling module for downsampling and upsampling interaction sequence on the temporal axis, respectively. The system represents the 2D or 3D spatial relation of human and objects from the detection results in video data as a graph. To learn the human-object relations, a new attention graph convolutional network is trained to extract condensed information from the graph representation. To segment action into sub-actions, a novel temporal pyramid pooling module is proposed, which upsamples compressed features back to the original time scale and classifies actions per frame. We explore various attention layers, namely spatial attention, temporal attention and channel attention, and combine different upsampling decoders to test the performance on action recognition and segmentation. We evaluate our model on two challenging datasets in the field of human-object interaction recognition, i.e. Bimanual Actions and IKEA Assembly datasets. We demonstrate that our classifier significantly improves both framewise action recognition and segmentation, e.g., F1 micro and F1@50 scores on Bimanual Actions dataset are improved by $4.3\%$ and $8.5\%$ respectively.
関連論文
- 身体駆動型統合マルチオブジェクト物理による人物-物体インタラクション生成人物-物体インタラクション
