The video shows us how our eyes can make us think that a sound is coming from a particular source, which is actually produced by a different object. This is called the ventriloquism effect (Goldstein, 2010). The movement of the puppet’s mouth creates the impression that the voice is coming from it. But in reality, the speaker is the man holding the puppet, the ventriloquist. This effect is brought about by the fact that our senses do not work in isolation. Our eyes and ears, as well as the others communicate with each other and integrate whatever information they get to give us a unified experience.
I found out recently that the ventriloquism effect can be based not only on visual information but can also be brought by tactile stimulation. A so-called audio-tactile ventriloquism effect was observed in an experiment where the perceived location of a sound become biased toward the center after a tactile stimuli is applied to centrally located fingertips (Bertelson & Aschersleben, 1998 as cited in Bruns & Röder, 2010). One practical example I could think of is the trick that friends usually do that goes when someone from behind calls out your name and taps on your shoulder. Your tendency is to look at the person in the direction where you felt the tap and not from where the voice is coming from. So you turn around only to find out that no one is there because your friend is actually at the opposite side.
Bruns & Röder (2010) studied the audio-tactile ventriloquism effect further by testing if hand posture influences the magnitude and the direction of this effect. They want to find out if it functions in an anatomically centered coordinate system, which relies on the spread of activation within the same hemisphere. If the hands were crossed, stimulating the right hand positioned in the left side would influence the sound localization towards the right. However, it could also function in an external coordinate system where the external location of the hand directs sound localization. For example, if the hands were crossed, stimulating the right hand positioned in the left side would bias the sound localization towards the left.
Ten female and one male participant took part in the experiment with 1,800 trials where they had to report the perceived location of an auditory stimuli presented from the center, right and left sides. They were seated in a very dark room and their hands alternate between the crossed and uncrossed- hands condition. Their right and left index fingers were positioned over the right and left stimulators on the desk, respectively, in the uncrossed-hands condition. In the crossed hands condition, their right and left index fingers were positioned over the left and the right stimulators, respectively. There were nine stimulus conditions wherein there are 3 sound only conditions (from the left, right or center locations), 2 tactile stimulation only condition (either in the left or right index finger), and 4 conditions with the combination of the previous conditions (sounds from the center paired with either the left and right hand stimulation, sound from the right pared with a left hand stimulation, and vise versa). They had to respond to where they think the sound is coming from by pressing the foot pedals located under the desk.
It was found out that audio-tactile ventriloquism effect depends on the physical distance between stimuli. Sound localization bias towards the direction of the tactile stimuli is greater in larger audio-tactile spatial discrepancies (in sound from the right pared with a left hand stimulation, and vise versa) relative to those with small discrepancies (sounds from the center paired with either the left and right hand stimulation). It primarily operates in the external coordinate system. Furthermore, when the participants adopted the crossed hands condition, the ventriloquism effect was reduced.
Bruns & Röder (2010) reason that “it might be speculated that a less precise spatial representation of the tactile stimuli in the crossed-hands condition resulted in the reduced influence of tactile stimuli on auditory localization found in the present experiment.” This is in line with other crossmodal interactions that depend on the relative reliability of sensory inputs that make up our multi-sensory experience.
Finally, this study reminds us over and over again that our senses do not work alone. They communicate and interact with each other to come-up with a holistic experience. In some situations, a particular sense modality appears superior to the other like in the ventriloquism effect where we rely more on either on visual or in tactile stimulation relative to the auditory signals. But, it does not mean that our sense of hearing is less important and there are events wherein sounds color our visual or tactile experience. It just happens that whatever sensory input is more reliable is utilized.
___________________________________________________
Bruns, P. & Röder, B. (2010). Tactile capture of auditory localization is modulated by hand posture. Experimental Psychology, 57, 4, 267-274. Retrieved from EBSCOhost.
Goldstein, E.B. (2010). Sensation and perception (8th ed.). California, USA: Wadsworth.
http://www.youtube.com/watch?v=H6S5KCverSA&feature=related
0 comments:
Post a Comment