手術文脈の推論とジェスチャーへの変換に向けて
Towards Surgical Context Inference and Translation to Gestures
手術画像のセグメンテーションから道具と対象の位置関係で手術文脈を検出し、FSMやLSTMでジェスチャー記録を自動生成する手法を提案した。
著者: Kay Hutchinson, Zongyu Li, Ian Reyes, Homa Alemzadeh
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
Manual labeling of gestures in robot-assisted surgery is labor intensive, prone to errors, and requires expertise or training. We propose a method for automated and explainable generation of gesture transcripts that leverages the abundance of data for image segmentation. Surgical context is detected using segmentation masks by examining the distances and intersections between the tools and objects. Next, context labels are translated into gesture transcripts using knowledge-based Finite State Machine (FSM) and data-driven Long Short Term Memory (LSTM) models. We evaluate the performance of each stage of our method by comparing the results with the ground truth segmentation masks, the consensus context labels, and the gesture labels in the JIGSAWS dataset. Our results show that our segmentation models achieve state-of-the-art performance in recognizing needle and thread in Suturing and we can automatically detect important surgical states with high agreement with crowd-sourced labels (e.g., contact between graspers and objects in Suturing). We also find that the FSM models are more robust to poor segmentation and labeling performance than LSTMs. Our proposed method can significantly shorten the gesture labeling process (~2.8 times).
関連論文
- CTを用いた患者特異的シミュレーションと接触を考慮した人工内耳挿入のロボット計画手術支援ロボティクス
- 腹腔鏡手術における主意図予測のための手術活動文法の活用手術支援ロボティクス
- SurgeMOD: 画像空間の組織運動を視覚ベースの手術力に変換する手術支援ロボティクス
- 手術技術の違いに頑健な手術状態推定のための不変表現学習手術支援ロボティクス
- 姿勢が重要:低侵襲手術のための6自由度自律カメラ移動手術支援ロボティクス
- 学習ベースの深度知覚を用いた腹腔鏡最適視野制御のデータ駆動型統合フレームワーク手術支援ロボティクス
