対数尤度比融合によるドローン・移動ロボット遠隔操作のための解釈可能なマルチモーダルジェスチャ認識
Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion
両手首のApple Watchの慣性データと自作グローブの静電容量センサを対数尤度比で後期融合し、遮蔽や照明変化に頑健で解釈可能なジェスチャ認識を実現した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Seungyeol Baek, Jaspreet Singh, Lala Shakti Swarup Ray, Hymalai Bello, Paul Lukowicz, Sungho Suh
分類: cs.RO, cs.AI
原文アブストラクト
Human operators are still frequently exposed to hazardous environments such as disaster zones and industrial facilities, where intuitive and reliable teleoperation of mobile robots and Unmanned Aerial Vehicles (UAVs) is essential. In this context, hands-free teleoperation enhances operator mobility and situational awareness, thereby improving safety in hazardous environments. While vision-based gesture recognition has been explored as one method for hands-free teleoperation, its performance often deteriorates under occlusions, lighting variations, and cluttered backgrounds, limiting its applicability in real-world operations. To overcome these limitations, we propose a multimodal gesture recognition framework that integrates inertial data (accelerometer, gyroscope, and orientation) from Apple Watches on both wrists with capacitive sensing signals from custom gloves. We design a late fusion strategy based on the log-likelihood ratio (LLR), which not only enhances recognition performance but also provides interpretability by quantifying modality-specific contributions. To support this research, we introduce a new dataset of 20 distinct gestures inspired by aircraft marshalling signals, comprising synchronized RGB video, IMU, and capacitive sensor data. Experimental results demonstrate that our framework achieves performance comparable to a state-of-the-art vision-based baseline while significantly reducing computational cost, model size, and training time, making it well suited for real-time robot control. We therefore underscore the potential of sensor-based multimodal fusion as a robust and interpretable solution for gesture-driven mobile robot and drone teleoperation.
関連論文
- AeroIntent: UAVミッション意図認識のための同期マルチモーダルベンチマークマルチモーダル認識
- MultiFly: 実世界マルチモーダル航空データセットと注釈効率の良いラベル転送およびクロスモーダル意味整合性マルチモーダル認識
- GlassFormer: レーダーと深度の融合によるリアルタイムガラスセグメンテーションの学習マルチモーダル認識
- 照明条件に応じてRGBと赤外線を適応融合する自律追尾タレットシステムの評価マルチモーダル認識
- OmniUnet: RGB・深度・熱画像を用いた惑星探査ローバー向け非構造地形セグメンテーションのマルチモーダルネットワークマルチモーダル認識
- 敵対的特徴分離による頑健な手術ワークフロー認識のためのマルチモーダルグラフ表現学習マルチモーダル認識
