Abstract:
Few domains are as central to cognitive science as language and visual perception. While they are typically studied independently, at some level, they must connect. How? In my talk, I'll present research demonstrating that language and vision share core aspects of representational format and content, using events and spatial relations as key examples. First, I will present evidence that visual perception automatically encodes relational elements such as Agent, Patient, support, and containment, in ways that instantiate principles typically associated with linguistic representationsabstraction, categoricality, and structure. Such properties point to an intriguing possibility: Vision, like language, may be compositional. With this in mind, Ill explore the psychophysics of visual scene composition, finding that the mind builds relational representations (e.g., a vase on a table) sequentially and in a canonical order, respecting each element's role in the relation. These findings offer insight into how structured representations emerge in the mind and highlight parallels with language comprehension, where structure is built hierarchically despite linear, sequential input. Finally, I'll discuss how these parallels might inform our understanding of how languages express relations, discussing some recent work exploring linguistic and nonlinguistic constraints on what verb meanings can get lexicalized.
Bio: Dr. Alon Hafri