共有潜在変数によるロボット動作学習とLLMの統合:感覚運動注意と言語ベース回帰
Sensorimotor Attention and Language-based Regressions in Shared Latent Variables for Integrating Robot Motion Learning and LLM
ロボット動作学習モデルとLLMを共有潜在変数で接続し、感覚運動注意点とタスク言語指示からの予測誤差でパラメータを更新することで、位置と言語指示への汎化性能を高める手法を提案。
著者: Kanata Suzuki, Tetsuya Ogata
分類: cs.RO
原文アブストラクト
In recent years, studies have been actively conducted on combining large language models (LLM) and robotics; however, most have not considered end-to-end feedback in the robot-motion generation phase. The prediction of deep neural networks must contain errors, it is required to update the trained model to correspond to the real environment to generate robot motion adaptively. This study proposes an integration method that connects the robot-motion learning model and LLM using shared latent variables. When generating robot motion, the proposed method updates shared parameters based on prediction errors from both sensorimotor attention points and task language instructions given to the robot. This allows the model to search for latent parameters appropriate for the robot task efficiently. Through simulator experiments on multiple robot tasks, we demonstrated the effectiveness of our proposed method from two perspectives: position generalization and language instruction generalization abilities.
