Ventriloquist Effect

Ventriloquists never fail to fascinate me with their ability to make inanimate objects such as a puppet to speak and communicate with the audience. Everytime I see one, it makes me want to try what they are doing only to make a fool out of myself realizing how difficult it is to perfect that kind of act. To see what I mean, better check out this video showing a talented Filipino ventriloquist.


The video shows us how our eyes can make us think that a sound is coming from a particular source, which is actually produced by a different object. This is called the ventriloquism effect (Goldstein, 2010). The movement of the puppet’s mouth creates the impression that the voice is coming from it. But in reality, the speaker is the man holding the puppet, the ventriloquist. This effect is brought about by the fact that our senses do not work in isolation. Our eyes and ears, as well as the others communicate with each other and integrate whatever information they get to give us a unified experience.

I found out recently that the ventriloquism effect can be based not only on visual information but can also be brought by tactile stimulation. A so-called audio-tactile ventriloquism effect was observed in an experiment where the perceived location of a sound become biased toward the center after a tactile stimuli is applied to centrally located fingertips (Bertelson & Aschersleben, 1998 as cited in Bruns & Röder, 2010). One practical example I could think of is the trick that friends usually do that goes when someone from behind calls out your name and taps on your shoulder. Your tendency is to look at the person in the direction where you felt the tap and not from where the voice is coming from. So you turn around only to find out that no one is there because your friend is actually at the opposite side.

Bruns & Röder (2010) studied the audio-tactile ventriloquism effect further by testing if hand posture influences the magnitude and the direction of this effect. They want to find out if it functions in an anatomically centered coordinate system, which relies on the spread of activation within the same hemisphere. If the hands were crossed, stimulating the right hand positioned in the left side would influence the sound localization towards the right. However, it could also function in an external coordinate system where the external location of the hand directs sound localization. For example, if the hands were crossed, stimulating the right hand positioned in the left side would bias the sound localization towards the left.

Ten female and one male participant took part in the experiment with 1,800 trials where they had to report the perceived location of an auditory stimuli presented from the center, right and left sides. They were seated in a very dark room and their hands alternate between the crossed and uncrossed- hands condition. Their right and left index fingers were positioned over the right and left stimulators on the desk, respectively, in the uncrossed-hands condition. In the crossed hands condition, their right and left index fingers were positioned over the left and the right stimulators, respectively. There were nine stimulus conditions wherein there are 3 sound only conditions (from the left, right or center locations), 2 tactile stimulation only condition (either in the left or right index finger), and 4 conditions with the combination of the previous conditions (sounds from the center paired with either the left and right hand stimulation, sound from the right pared with a left hand stimulation, and vise versa). They had to respond to where they think the sound is coming from by pressing the foot pedals located under the desk.

It was found out that audio-tactile ventriloquism effect depends on the physical distance between stimuli. Sound localization bias towards the direction of the tactile stimuli is greater in larger audio-tactile spatial discrepancies (in sound from the right pared with a left hand stimulation, and vise versa) relative to those with small discrepancies (sounds from the center paired with either the left and right hand stimulation). It primarily operates in the external coordinate system. Furthermore, when the participants adopted the crossed hands condition, the ventriloquism effect was reduced.

Bruns & Röder (2010) reason that “it might be speculated that a less precise spatial representation of the tactile stimuli in the crossed-hands condition resulted in the reduced influence of tactile stimuli on auditory localization found in the present experiment.” This is in line with other crossmodal interactions that depend on the relative reliability of sensory inputs that make up our multi-sensory experience.

Finally, this study reminds us over and over again that our senses do not work alone. They communicate and interact with each other to come-up with a holistic experience. In some situations, a particular sense modality appears superior to the other like in the ventriloquism effect where we rely more on either on visual or in tactile stimulation relative to the auditory signals. But, it does not mean that our sense of hearing is less important and there are events wherein sounds color our visual or tactile experience. It just happens that whatever sensory input is more reliable is utilized.

___________________________________________________

Bruns, P. & Röder, B. (2010). Tactile capture of auditory localization is modulated by hand posture. Experimental Psychology, 57, 4, 267-274. Retrieved from EBSCOhost.

Goldstein, E.B. (2010). Sensation and perception (8th ed.). California, USA: Wadsworth.

http://www.youtube.com/watch?v=H6S5KCverSA&feature=related







  • Digg
  • Del.icio.us
  • StumbleUpon
  • Reddit
  • RSS

Party Out Loud


Every Friday or Saturday, I try to go with my friends to have a good time in ending that stressful week. We usually have dinner and kwentuhan session when we are together. But, as what most college students do, we also go to parties with free flowing drinks and non-stop music. Just feeling the good vibes of the music and the people around take my stress away and make me ready to face another loaded week ahead.

College parties are so different from other parties I have experienced in high school; so much fun than the latter. There, we talk to people we do not know, make new friends, talk our emotions out, express oneself without inhibition, and yet we do not forget what we usually do: kwentuhan.

Even the music is very loud and all the people are shouting, we still and somehow find a way to tell stories to another. Surprisingly, we grasp each other’s stories at least the gist. Scanning through terms in Perception, I found out that this phenomenon is called the Cocktail Party Effect. It is an impressive ability, sometimes under-appreciated, to listen to a talker even with the presence of other conversation and background music. Sometimes, it is used for eavesdropping a conversation while neglecting others and the music.

Lisa Stifelman (1994) conducted an experiment involving Cocktail Party Effect. She wanted to build auditory interfaces that would lessen the amount of time in listening. By presenting multiples of audio streams at the same time, the listener would focus on one stream but could get interesting and significant information through overhearing other.

In this study, the subjects were presented with multiple channels of audio and they were asked to perform listening comprehension and target monitoring at the same time. While listening, the subject should identify target words in audios played simultaneously. As a result of this study, Stifelman found out that as the number of channels increased, the performance of the two tasks decreased.

This shows that one cannot listen to more than one sound stimulus at a time. If possible, it would not be that accurate like what this study showed. And so, this just tells that this effect is very interesting and beneficial. Some are not aware of this ability and so, they do not try to listen even there is a loud sound playing in the background. But, we must not overuse this ability; or should I say, we must not use this to eavesdrop other people’s private conversation. Anyway, you cannot do that at the same time listening with your own friends because what you hear might not be accurate anymore. So, you would not totally grasp any at all.

Stifelman, L. J. (1994). The Cocktail Party Effect in Auditory Interfaces: A Study of Simultaneous Presentation. MIT Media Laboratory Technical Report.

Dean, J. (2009). The Cocktail Party Effect. PsyBlog: Understand YourMmind. Retrieved from http://www.spring.org.uk/2009/03/the-cocktail-party-effect.php

  • Digg
  • Del.icio.us
  • StumbleUpon
  • Reddit
  • RSS

When you sing nothing at all

I have been in a choral group since I was 8 years old. My love for music developed ever since and the thirst to learn more became more evident in the years that followed. We were taught how to discriminate between notes, how to get the right pitch, how to follow beats and rhythms and how to blend in with the other voices. It was a tedious task at first but as you go along, the knowledge becomes implicit to the point where you can no longer verbally explain how you were able to do it. You just listen and you just know.


In relation to this, a study done by Krumhansi and Iverson (1992), researched on the perceptual interaction between pitch and timbre. Although they were much focused on that of the musical instruments', i find it useful to know that the vocal chords are considered to be a musical instrument as well and that we can apply the voice as the output of that instrument. 


The participants were all from Cornell University, each with a requirement of having studied at least one musical instrument for a minimum of 5 years ending not more than 3 years before participating in the study. The procedure was to simply categorize the presented tones in several ways. After hearing a tone from the speaker they will proceed to reading the categories on the screen (e.g. Category 1 = high trumpret, Category 2 = low piano, etc.) and they were instructed to press the number of the category from which they think will best fit the tone. 


The results showed that subjects could not attend to the pitch of a tone without being influenced by its timbre and could not attend to the pitch of a tone without being influenced by its pitch. The interaction effects were symmetrical. Now I understand why not all singers can have a duet or a solo because it depends on the vocal timbre as every instruments' vary. So the next time that Sir Raul asks us the choir to sing as one (as to maintain a single timbre per voice), I will not be annoyed because this affects the precision from which the other aspects of  the music are perceived. 


Krumhansl, C. L., & Iverson, P. (1992). Perceptual interactions between musical pitch and timbre. Journal of Experimental Psychology: Human Perception and Performance, 18(3), 739-751. doi:10.1037/0096-1523.18.3.739

  • Digg
  • Del.icio.us
  • StumbleUpon
  • Reddit
  • RSS

Ride with Me


Driving is my passion. I’ve started driving since I was in third year high school. I love driving but city driving in Metro Manila is really horrible. I feel like I’m always in danger while sharing the street with PUVs. Others say that the key to a safe driving is defensive driving. For me, that doesn’t work in the streets of Metro Manila. I should say that a lot of PUV drivers are really aggressive and abusive. I’m sure you also have your own story of how these big monsters along EDSA almost scared you to death but more than that, how dangerous is it when we go out of the metro and pass through highways.

A study was done by Rob Gray and David Regan 10 years ago about risky driving behaviours. They studied the consequences of motion adaptation for visually guided motor action outside the laboratories. Overtaking and passing maneuvers in highways was one of their focuses. As a driver, one of the latest skills that I learned was overtaking. Especially in the Philippines, overtaking is very dangerous. The streets are kind of narrow and there are a lot of cars. On the other hand, overtaking is inevitable in the Philippines. PUV doesn’t have loading and unloading zones which cause them to always slow down and stop. Although patience is a virtue, there are still days that you need to rush in order to come in time for a class or a meeting. In general, a lot of major accidents in the road happen because of overtaking and passing maneuvers.

Although the study was done in the United States, which roads are completely different in the Philippines, they found that overtaking is really risky especially when the driver is just staring at a blank space. This may be due to fatigue effect if there is prolonged driving or because the visual perception of the driver overestimates the motion of the vehicle. You might wonder how the experimenters came up with this generalization. Their experiment was a bit costly because they used fixed-base driving simulator composed of the frontal two-thirds of a Nissan 240SX convertible and a wide-field-of-view (60° horizontal X 40° vertical) display of a simulated driving scene. Octane workstation (Silicon Graphics Inc.) fixed and edited the scene. It was projected onto a wall 3.5 m in front of the driver with a Barco 800G projector and was continually changed at an average rate of 15 frames/s in correspondence with the movement of the car. (I’m not really a techie person but I just can imagine how elegant their setup is) I really wish that I have been a participant in this study. Each participant is given 10 minutes to be comfortable with the system before they were asked to perform the overtaking trial. The instruction was pretty easy: overtake the car just like as how you’d do in a real highway. With this, they came up with a conclusion that adaptation to retinal image expansion has a dramatic effect on overtaking maneuvers. They also found out that observers drove significantly faster in the adaptation condition than in either of the baseline conditions. This means that drivers drive faster when they see an empty road, perceiving less visual motion.

This may explain why there are a lot of accidents in provincial buses in the Philippines nowadays. There might be a fatigue effect and they might not see much road signs which sends visual motion to them. They tend to overestimate that they could overtake anytime without any collisions. Maybe this study can call the attention of the MMDA and DPWH in dealing with accidents. I hope that there’s still a chance for the Philippines to be a safer place for driving. And in addition to that, I hope that there will be more attempts to study the driver’s visual perception in the Philippines’ setting.



Gray, R and Megan, D (2000) Risky Driving Behavior: A Consequence of Motion Adaptation for Visually Guided Motor Action. Journal of Experimental Psychology: Human Perception and Performance 2000, Vol. 26, No. 6, 1721-1732. Retrieved from http://www.apa.org/pubs/journals/releases/xhp2661721.pdf

  • Digg
  • Del.icio.us
  • StumbleUpon
  • Reddit
  • RSS

sound localization in autism

During the discussion on visual attention, it has been mentioned that autistic patients tend not to look at the salient stimuli in the environment, but instead at the background scene. In addition to visual inattention, Teder-Salejarvi, Pierce, Courchesne, & Hillyard (2005) discussed that people with autism have other sensory abnormalities and these include the auditory domain.

Parallel to the visual inattention, the hypothesized reason for this is their difficulty to attend to sound signals amidst noisy environment. They are either hyposensitive to acoustic cues to the point of completely ignoring sounds (Dawson, et.al., 1998: Kemper, et. al., 1998; cited in Teder-Salejarvi, et. al., 2005) or hypersensitive to such signals (Baranek, et. al., 1997; Bettison, 1996; cited in Teder-Salejarvi, et. al., 2005). Temple Grandin, a person with autism, pictured her experience of confusion when two people are talking at the same time. She described her ears as a microphone that picks up all sounds, all in equal intensity making it impossible to screen out the background noise and to understand the speech of other people in such noisy place. This failure to accurately tune spatial attention to a single sound, or in some cases, tuning too broadly, according to Teder-Salejarvi and colleagues, impedes orienting which then decreases accuracy of target detection and causes information to be missed. As a consequence, and as Courchesne, et.al. (1994; cited in Teder-Salejarvi, et. al., 2005) reports, autism is also related to the “developmental failure to engage in joint social attention” (p. 221).

Since no studies had tested the hypothesis that sound source attribution and selection may be involved in autism using neurofunctional techniques, Teder-Salejarvi, et. al. (2005) used event-related potential (ERP) measures to compare and characterize the tuning auditory spatial attention of adults with autism and those who are “healthy” (p.222), the latter as the control group. According to literature (i.e.Hansen, et. al., 1980; Hillyard, et. al., 1995; Teder, et. al., 1993; cited in Teder-Salejarvi, et. al., 2005), recorded ERP from normal subjects had revealed that enhanced negative wave in the auditory cortex (N1) is usually elicited when sounds are being attended to. As more attention is driven to the sound— thus in consequence the more its accuracy is improved— the more that the amplitude of the negative wave in the auditory cortex also increases. On the other hand, target detection involves a longer latency of ERP which is labeled P3 (Teder-Salejarvi, et. al., 2005) which according to Martineau, et.al. (1984) and Picton, et. al. (1999; cited in Teder-Salejarvi, et. al., 2005) is associated with the updating of working memory once target is being recognized.

To explore the mystery behind, that is whether the autistic patients would “display abnormalities in the selective tuning of attention to one sound source in the presence of multiple competing sources” (p.222), Teder-Salejarvi, et. al. (2005) had planned the research well and measured related factors in the experiment and employed necessary measures such as EEG recording, sensory discrimination testing specifically frequency and spatial discrimination tasks, topographical mapping, MRI anatomical measurements, and of course, the ERP analysis. Details of each of these are described in their paper, also each of these are essential to investigate where the problem, in case hypothesis is supported, comes from. For instance, through MRI anatomical measurements, this study has revealed that participants with autism had 7% less temporal gray matter compared to normal (Teder-Salejarvi, et. al., 2005); in some studies reduced volume of the left planum temporal (Rojas, et. al., 2001; cited in Teder-Salejarvi, et. al., 2005), and an abnormality in the white matter of the temporal lobes (Bamea-Goraly, et. al., 2004; cited in Teder-Salejarvi, et. al., 2005).

Before discussing the results further, just to describe the method of the study, there were 8 speakers that produce sounds altogether. The first one is place in the front center (0) of the participant and the three succeeding speakers are placed one after another, 6 apart from each other. While the rest of the speakers at the right periphery of the listener (72 78 84 and 90). Noise burst were present with an intensity level of 76 dB. The onset asynchronies of stimulus were varied between 90 and 270ms (Teder-Salejarvi, et. al., 2005). As for the procedure, the participants were asked to attend, either to the central speaker or to the rightmost one; they have to ignore stimuli coming from other speakers and press a button to an infrequent high-pitched burst of noise, which is the target stimulus. Ten rounds of 1092 stimuli were presented in each condition! Teder-Salejarvi, et. al. (2005) did not also forget the architectural design’s influence to perceiving sound and conducted the experiment in an “acoustically shielded room” (p.223).

Supporting the hypothesis, the results of the study showed that adults with autism disorder have diminished ability to attend to a significant stimulus from its sound source among the many outer sources of sound in the environment. It has also been found that autistic adults have small “fronto-centrally distributed N1” (p.226) while P3 wave were broadly distributed compared to the control group who had negative enhancement of the N1 and a widely distributed P3 (Teder-Salejarvi, et. al., 2005). The amplitude of the N1 waves produced either by the center or periphery of the listener, did not differ significanlty ( y[6]= 1.19). The P3’s amplitude, based on the locations of the stimulus, whether it was placed on the S1 or S8, have no significant difference.

The study shows that auditory-related abnormalities in autism might be due to the impairment in sound source attribution (Teder-Salejarvi, et. al., 2005). These abnormalities related to auditory, aside from hypersensitivity and hyposensitivity to sounds, may also include confusion, aversive reactions to sounds, and abnormality in orienting and shifting one's attention as a response to sound signals. Finally, the results of the study implies that difficulty of sound localization might be a factor as to how autistic infants react to and learn, both from the social and non-social sounds. Also if this difficulty of autistic patients may have is apparent in a noisy environment, a simplifier and less noisy setting may lessen such difficulty.

Reference:

Teder-Salejarvi, W. A, Pierce, K. L., Courchesne, E., & Hillyard, A. A. et. al. (2005). Auditory spatial localization and attention deficits in autistic adults. Cognitive Brain Research. 23. 221-234. Retrieved from Microsoft Academic Research.

  • Digg
  • Del.icio.us
  • StumbleUpon
  • Reddit
  • RSS