タスク対称性を考慮したロボット政策学習のための対称性活用法
Symmetry Considerations for Learning Task Symmetric Robot Policies
目標条件付きロボットタスクにおいて、データ拡張とミラー損失関数を用いて対称性不変性を深層強化学習に組み込む手法を提案し、四足歩行の箱登りや巧みな操作タスクで収束の高速化と性能向上を実証した。
著者: Mayank Mittal, Nikita Rudin, Victor Klemm, Arthur Allshire, Marco Hutter
分類: cs.RO, cs.AI
原文アブストラクト
Symmetry is a fundamental aspect of many real-world robotic tasks. However, current deep reinforcement learning (DRL) approaches can seldom harness and exploit symmetry effectively. Often, the learned behaviors fail to achieve the desired transformation invariances and suffer from motion artifacts. For instance, a quadruped may exhibit different gaits when commanded to move forward or backward, even though it is symmetrical about its torso. This issue becomes further pronounced in high-dimensional or complex environments, where DRL methods are prone to local optima and fail to explore regions of the state space equally. Past methods on encouraging symmetry for robotic tasks have studied this topic mainly in a single-task setting, where symmetry usually refers to symmetry in the motion, such as the gait patterns. In this paper, we revisit this topic for goal-conditioned tasks in robotics, where symmetry lies mainly in task execution and not necessarily in the learned motions themselves. In particular, we investigate two approaches to incorporate symmetry invariance into DRL -- data augmentation and mirror loss function. We provide a theoretical foundation for using augmented samples in an on-policy setting. Based on this, we show that the corresponding approach achieves faster convergence and improves the learned behaviors in various challenging robotic tasks, from climbing boxes with a quadruped to dexterous manipulation.
