LyRN: 単眼視からのリアルタイム閉ループ制御によるリーチング
LyRN (Lyapunov Reaching Network): A Real-Time Closed Loop approach from Monocular Vision
制御リアプノフ関数を深層学習で予測し、単眼RGBカメラと関節角から複数物体の中から最適な対象へリアルタイムにリーチング・把持する閉ループ制御手法を提案。シミュレーションのみで学習し実世界で90.3%の把持成功率を達成。
著者: Zheyu Zhuang, Xin Yu, Robert Mahony
分類: cs.RO
原文アブストラクト
We propose a closed-loop, multi-instance control algorithm for visually guided reaching based on novel learning principles. A control Lyapunov function methodology is used to design a reaching action for a complex multi-instance task in the case where full state information (poses of all potential reaching points) is available. The proposed algorithm uses monocular vision and manipulator joint angles as the input to a deep convolution neural network to predict the value of the control Lyapunov function (cLf) and corresponding velocity control. The resulting network output is used in real-time as visual control for the grasping task with the multi-instance capability emerging naturally from the design of the control Lyapunov function. We demonstrate the proposed algorithm grasping mugs (textureless and symmetric objects) on a table-top from an over-the-shoulder monocular RGB camera. The manipulator dynamically converges to the best-suited target among multiple identical instances from any random initial pose within the workspace. The system trained with only simulated data is able to achieve 90.3% grasp success rate in the real-world experiments with up to 85Hz closed-loop control on one GTX 1080Ti GPU and significantly outperforms a Pose-Based-Visual-Servo (PBVS) grasping system adapted from a state-of-the-art single shot RGB 6D pose estimation algorithm. A key contribution of the paper is the inclusion of a first-order differential constraint associated with the cLf as a regularisation term during learning, and we provide evidence that this leads to more robust and reliable reaching/grasping performance than vanilla regression on general control inputs.
関連論文
- 3D特徴量に基づくビジュアルサーボ:ロボット挿入タスクの設計と実装ビジュアルサーボ/マニピュレーション
- 双四元数を用いた移動物体把持のためのビジュアルサーボビジュアルサーボ/マニピュレーション
