SPEAR: 写実的な具現化AI研究のためのシミュレータ
SPEAR: A Simulator for Photorealistic Embodied AI Research
Unreal EngineをPythonから制御できる高速・高機能な写実シミュレータを提案し、多様なエージェント制御やデータ生成を実現した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Mike Roberts, Renhan Wang, Rushikesh Zawar, Rachith Dey-Prakash, Quentin Leboutet, Stephan R. Richter, Matthias Müller, German Ros, Rui Tang, Stefan Leutenegger, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Vladlen Koltun
分類: cs.CV, cs.AI, cs.GR, cs.LG, cs.RO
原文アブストラクト
Interactive simulators have become powerful tools for training embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame. We demonstrate the utility of SPEAR through a diverse collection of example applications: controlling multiple embodied agents with distinct action spaces (e.g., humans, cars, and robots) across several in-the-wild UE projects; rendering photorealistic city-scale environments; manipulating UE's procedural content generation systems; rendering synchronized multi-view images of detailed human faces; coordinating an interactive co-simulation with the MuJoCo physics simulator; and editing scenes with natural language via an AI coding assistant.
関連論文
- 詳細メッシュ対応の接触リッチ相互作用向け軽量シミュレータLodosシミュレーション
- Ostrich: 微分可能ダイナミクスにおける剛直な接触を跨ぐ大きな歩幅シミュレーション
- Agri-Sim: 温室ロボティクスにおける具現化知能評価のための農業シミュレーションプラットフォームシミュレーション
- 浅海域の写実性を超えて：深海ロボティクスのための物理・センサ接地シミュレーションシミュレーション
- NVIDIA Cosmos-H-Dreams: 手術ロボットのためのリアルタイム生成物理シミュレーションシミュレーション
- CRESSim-Neo: 手術ロボティクスとロボット学習のためのバッチ処理GPUシミュレーションエンジンシミュレーション
