In my work I have attempted to determine the existence, the nature and the mechanism underlying crossmodal integration of multisensory non-speech auditory and visual stimuli based on their content correspondence. Using both behavioral and brain imaging methods, I explored the crossmodal integration based on feature correspondence between auditory pitch of sound and the visual features comprising vertical location, size, spatial frequency and contrast. In a series of psychophysical experiments I found natural mappings between the pitch of sounds and the visual features of vertical location, size and spatial frequency. The facilitatory effects observed during congruent bimodal stimulation hint at the automatic nature of these cross-modal interactions as well as their independence of attention.
In the fMRI study I found that the participating sensory cortices as well as multimodal sites (superior temporal sulcus and superior temporal gyrus) play a role in these crossmodal interactions. The perceptual gains are realized by amplification of the neuronal response in the participating sensory cortices, which is modulated by the congruency between the two features.