微分可能なセマンティックマッピングと計画による自律ナビゲーションの逆強化学習
Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning
距離とセマンティックカテゴリの観測から専門家のコスト関数を逆強化学習で推定し、新しい環境のセマンティック配置にも汎化できるナビゲーション手法を提案した。CARLAで交通ルールに従う走行を学習できることを示した。
著者: Tianyu Wang, Vikas Dhiman, Nikolay Atanasov
分類: cs.LG, cs.RO
原文アブストラクト
This paper focuses on inverse reinforcement learning for autonomous navigation using distance and semantic category observations. The objective is to infer a cost function that explains demonstrated behavior while relying only on the expert's observations and state-control trajectory. We develop a map encoder, that infers semantic category probabilities from the observation sequence, and a cost encoder, defined as a deep neural network over the semantic features. Since the expert cost is not directly observable, the model parameters can only be optimized by differentiating the error between demonstrated controls and a control policy computed from the cost estimate. We propose a new model of expert behavior that enables error minimization using a closed-form subgradient computed only over a subset of promising states via a motion planning algorithm. Our approach allows generalizing the learned behavior to new environments with new spatial configurations of the semantic categories. We analyze the different components of our model in a minigrid environment. We also demonstrate that our approach learns to follow traffic rules in the autonomous driving CARLA simulator by relying on semantic observations of buildings, sidewalks, and road lanes.
