シールド付きマルチエージェント強化学習による連続体マニピュレータの安全制御
Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learning
2自由度ケーブル駆動連続体手術マニピュレータの制御に、行動集合境界を動的に変化させるシールド機構を組み合わせたマルチエージェント深層Qネットワークを適用し、外乱や衝突下でもサブミリ精度の追従を実現した。
著者: Guanglin Ji, Junyan Yan, Jingxin Du, Wanquan Yan, Jibiao Chen, Yongkang Lu, Juan Rojas, Shing Shin Cheng
分類: cs.RO, cs.AI, cs.MA
原文アブストラクト
Continuum robotic manipulators are increasingly adopted in minimal invasive surgery. However, their nonlinear behavior is challenging to model accurately, especially when subject to external interaction, potentially leading to poor control performance. In this letter, we investigate the feasibility of adopting a model-free multiagent reinforcement learning (RL), namely multiagent deep Q network (MADQN), to control a 2-degree of freedom (DoF) cable-driven continuum surgical manipulator. The control of the robot is formulated as a one-DoF, one agent problem in the MADQN framework to improve the learning efficiency. Combined with a shielding scheme that enables dynamic variation of the action set boundary, MADQN leads to efficient and importantly safer control of the robot. Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads, soft obstacles, and rigid collision, which are common interaction scenarios encountered by surgical manipulators. The controller was further proven to be effective in a miniature continuum robot with high structural nonlinearitiy, achieving trajectory tracking with submillimeter accuracy under external payload.
