日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

メタバース/遠隔操作arXiv:2407.16591

メタバースにおける人間操作者と遠隔デバイス間のリアルタイムインタラクション

Real-Time Interactions Between Human Controllers and Remote Devices in Metaverse

シェア:XThreadsFacebookLINEはてブBluesky

メタバース上で人間の操作者の動きを予測し、仮想モデルを描画と制御に分離して、描画の遅延と遠隔デバイス制御の誤差を同時に削減する強化学習フレームワークを提案した。

著者: Kan Chen, Zhen Meng, Xiangmin Xu, Changyang She, Philip G. Zhao

分類: cs.RO, cs.SY, eess.SY

原文アブストラクト

Supporting real-time interactions between human controllers and remote devices remains a challenging goal in the Metaverse due to the stringent requirements on computing workload, communication throughput, and round-trip latency. In this paper, we establish a novel framework for real-time interactions through the virtual models in the Metaverse. Specifically, we jointly predict the motion of the human controller for 1) proactive rendering in the Metaverse and 2) generating control commands to the real-world remote device in advance. The virtual model is decoupled into two components for rendering and control, respectively. To dynamically adjust the prediction horizons for rendering and control, we develop a two-step human-in-the-loop continuous reinforcement learning approach and use an expert policy to improve the training efficiency. An experimental prototype is built to verify our algorithm with different communication latencies. Compared with the baseline policy without prediction, our proposed method can reduce 1) the Motion-To-Photon (MTP) latency between human motion and rendering feedback and 2) the root mean squared error (RMSE) between human motion and real-world remote devices significantly.

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  3. 3newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

MINISFORUM(ミニPC)

MINISFORUM(ミニPC)の公式広告バナーMINISFORUM公式ストアを見る ↗
PR本紙発行元 EmplifAI