This article addresses issues in embodied sentence processing from a "cognitive neural systems" approach that combines analysis of the behavior in question, analysis of the known neurophysiological bases of this behavior, and the synthesis of a neuro-computational model of embodied sentence processing that can be applied to and tested in the context of human-robot cooperative interaction. We propose a Hybrid Comprehension Model that links compact propositional representations of sentences and discourse with their temporal unfolding in situated simulations, under the control of grammar. The starting point is a model of grammatical construction processing which specifies the neural mechanisms by which language is a structured inventory of mappings from sentence to meaning. This model is then "embodied" in a perceptual-motor system (robot) which allows it access to sentence-perceptual representation pairs, and interaction with the world providing the basis for language acquisition. We then introduce a "simulation" capability, such that the robot has an internal representation of its interaction with the world. The control of this simulator and the associated representations present a number of interesting "neuro-technical" issues. First, the "simulator" has been liberated from real-time. It can run without being connected to current sensory motor experience. Second, "simulations" appear to be represented at different levels of detail. Our paper provides a framework for beginning to address the questions: how does language and its grammar control these aspects of simulation, what are the neurophysiological bases, and how can this be demonstrated in an artificial yet embodied cognitive system.

, , ,
doi.org/10.1016/j.bandl.2009.07.001, hdl.handle.net/1765/17102
Brain and Language
,
Erasmus MC: University Medical Center Rotterdam

Madden, C., Hoen, M., & Dominey, P. F. (2010). A cognitive neuroscience perspective on embodied language for human-robot cooperation. Brain and Language, 112(3), 180–188. doi:10.1016/j.bandl.2009.07.001