視覚から力を読み取る：RGB-点群時系列畳み込みネットワークによる手術力予測
Learning to See Forces: Surgical Force Prediction with RGB-Point Cloud Temporal Convolutional Networks
内視鏡映像と点群データから組織変形を手がかりに手術中の力を推定する深層学習手法を提案し、ファントムと摘出肝臓で有効性を検証した。
著者: Cong Gao, Xingtong Liu, Michael Peven, Mathias Unberath, Austin Reiter
分類: cs.CV, cs.RO
原文アブストラクト
Robotic surgery has been proven to offer clear advantages during surgical procedures, however, one of the major limitations is obtaining haptic feedback. Since it is often challenging to devise a hardware solution with accurate force feedback, we propose the use of "visual cues" to infer forces from tissue deformation. Endoscopic video is a passive sensor that is freely available, in the sense that any minimally-invasive procedure already utilizes it. To this end, we employ deep learning to infer forces from video as an attractive low-cost and accurate alternative to typically complex and expensive hardware solutions. First, we demonstrate our approach in a phantom setting using the da Vinci Surgical System affixed with an OptoForce sensor. Second, we then validate our method on an ex vivo liver organ. Our method results in a mean absolute error of 0.814 N in the ex vivo study, suggesting that it may be a promising alternative to hardware based surgical force feedback in endoscopic procedures.
