MS-TCRNet：センサ拡張キネマティクスを用いた行動セグメンテーションのための多段時空間畳み込みリカレントネットワーク
MS-TCRNet: Multi-Stage Temporal Convolutional Recurrent Networks for Action Segmentation Using Sensor-Augmented Kinematics
キネマティクスデータの行動セグメンテーションのための多段時空間畳み込みリカレントネットワーク（MS-TCRNet）を提案し、幾何構造を利用したデータ拡張手法も導入して、手術手技データセットで最先端性能を達成した。
著者: Adam Goldbraikh, Omer Shubi, Or Rubin, Carla M Pugh, Shlomi Laufer
分類: cs.CV, cs.LG, cs.RO, eess.IV
原文アブストラクト
Action segmentation is a challenging task in high-level process analysis, typically performed on video or kinematic data obtained from various sensors. This work presents two contributions related to action segmentation on kinematic data. Firstly, we introduce two versions of Multi-Stage Temporal Convolutional Recurrent Networks (MS-TCRNet), specifically designed for kinematic data. The architectures consist of a prediction generator with intra-stage regularization and Bidirectional LSTM or GRU-based refinement stages. Secondly, we propose two new data augmentation techniques, World Frame Rotation and Hand Inversion, which utilize the strong geometric structure of kinematic data to improve algorithm performance and robustness. We evaluate our models on three datasets of surgical suturing tasks: the Variable Tissue Simulation (VTS) Dataset and the newly introduced Bowel Repair Simulation (BRS) Dataset, both of which are open surgery simulation datasets collected by us, as well as the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), a well-known benchmark in robotic surgery. Our methods achieved state-of-the-art performance.
