混雑環境における時相論理仕様下の連続制御のための深層強化学習による探索の克服
Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications
LTL仕様で与えられた複雑なタスクに対し、サンプリングベースの経路計画を組み込んだ報酬設計で探索を改善し、混雑環境での連続制御を深層強化学習で実現した。
著者: Mingyu Cai, Erfan Aasi, Calin Belta, Cristian-Ioan Vasile
分類: cs.RO, cs.LG
原文アブストラクト
Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) formula. To overcome the environmental challenge of exploration during training, we propose a novel path planning-guided reward scheme by integrating sampling-based methods to effectively complete goal-reaching missions. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-goal-reaching tasks that are solved in a distributed manner. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale cluttered environments. A video demonstration can be found on YouTube Channel: https://youtu.be/yMh_NUNWxho.
関連論文
- 動的障害物環境での安全飛行のための予測リスク誘導強化学習強化学習/ナビゲーション
- ManeuverNet: ダブルアクカーマン操舵ロボットの精密操縦のための報酬関数最適化を伴うSoft Actor-Criticフレームワーク強化学習/ナビゲーション
- 二重アクカーマン操舵ロボットの安全な機動のためのSoft Actor-Criticフレームワーク強化学習/ナビゲーション
- 分布強化学習に基づく自律水上艇の統合的意思決定と制御強化学習/ナビゲーション
- 渦流を横切るロボット魚の効率的ナビゲーション強化学習/ナビゲーション
- 自動運転車の安全クリティカル制御のための強化学習強化学習/ナビゲーション
