操作行動の意味論を学習する計算フレームワーク
Learning the Semantics of Manipulation Action
組み合わせ範疇文法とλ計算に基づき、操作行動の構文と意味を統一的に表現し、動画から確率的意味解析で学習する枠組みを提案した論文。
著者: Yezhou Yang, Yiannis Aloimonos, Cornelia Fermuller, Eren Erdal Aksoy
分類: cs.RO, cs.CL, cs.CV
原文アブストラクト
In this paper we present a formal computational framework for modeling manipulation actions. The introduced formalism leads to semantics of manipulation action and has applications to both observing and understanding human manipulation actions as well as executing them with a robotic mechanism (e.g. a humanoid robot). It is based on a Combinatory Categorial Grammar. The goal of the introduced framework is to: (1) represent manipulation actions with both syntax and semantic parts, where the semantic part employs $\lambda$-calculus; (2) enable a probabilistic semantic parsing schema to learn the $\lambda$-calculus representation of manipulation action from an annotated action corpus of videos; (3) use (1) and (2) to develop a system that visually observes manipulation actions and understands their meaning while it can reason beyond observations using propositional logic and axiom schemata. The experiments conducted on a public available large manipulation action dataset validate the theoretical framework and our implementation.
