HASARD: 身体性エージェントのための視覚ベース安全強化学習ベンチマーク
HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents
Doomを舞台に、一人称視覚のみで安全な意思決定を学ぶ強化学習エージェントを評価するベンチマークHASARDを提案し、難易度3段階と2種類の行動空間を用意して既存手法の性能と報酬・コストのトレードオフを分析した。
著者: Tristan Tomilin, Meng Fang, Mykola Pechenizkiy
分類: cs.AI, cs.CV, cs.LG, cs.RO
原文アブストラクト
Advancing safe autonomous systems through reinforcement learning (RL) requires robust benchmarks to evaluate performance, analyze methods, and assess agent competencies. Humans primarily rely on embodied visual perception to safely navigate and interact with their surroundings, making it a valuable capability for RL agents. However, existing vision-based 3D benchmarks only consider simple navigation tasks. To address this shortcoming, we introduce \textbf{HASARD}, a suite of diverse and complex tasks to $\textbf{HA}$rness $\textbf{SA}$fe $\textbf{R}$L with $\textbf{D}$oom, requiring strategic decision-making, comprehending spatial relationships, and predicting the short-term future. HASARD features three difficulty levels and two action spaces. An empirical evaluation of popular baseline methods demonstrates the benchmark's complexity, unique challenges, and reward-cost trade-offs. Visualizing agent navigation during training with top-down heatmaps provides insight into a method's learning process. Incrementally training across difficulty levels offers an implicit learning curriculum. HASARD is the first safe RL benchmark to exclusively target egocentric vision-based learning, offering a cost-effective and insightful way to explore the potential and boundaries of current and future safe RL methods. The environments and baseline implementations are open-sourced at https://sites.google.com/view/hasard-bench/.
