Photo: Daineko Natalia, Stockbym / Shutterstock
Believe nothing you see.
By NyRee Ausler — Written on Mar 20, 2023
The primary sense for humans is vision. We depend on it in every aspect of our lives and it is pretty straightforward.
Speech, on the other hand, is multimodal, meaning it relies on both auditory components and visual information. This is demonstrated every day in individuals with hearing disabilities.
They might be able to hear some parts of auditory speech, but also rely on lip movements, using those visual components to decipher speech, sound, and individual words.
But you don’t have to be hearing impaired to use auditory perception and visual influence to hear.
It comes second nature to us and is done automatically. We use auditory and visual information to understand what people are saying.
The perceptual phenomenon that shows how visual stimulus and sound work together to create a perception of speech is called the McGurk effect.
This personality test determines if you are able to hear with your eyes.
To test whether or not you use visual components along with what you hear to understand speech, Dan Mirea shared a TikTok video that tests your propensity to combine visual and auditory senses.
The test starts with a man saying “ba” over and over again. It’s evident the he is "mouthing" the sound simultaneously, so if you were to read his lips, you would come up with the correct sound he is making.
Next, Mirea adds another clip of the man making a similar sound, but if you are looking at him, you might notice that the movement of his lips is more conducive with pronunciation of a "V" or an "F."
If asked to reveal what you heard, you might say that he said “fa” or “va” when, in actuality, he made the exact same sound but changed his visual speech to throw you off.
You have just experienced the McGurk effect.
What is the McGurk effect?
The McGurk effect was first described in a 1976 paper entitled “Hearing Lips and Seeing Voices” by Harry McGurk and John MacDonald.
They found that a video with a different word spoken than mouthed resulted in multi-sensory illusion where they heard different words. This was believed to happen because of common visual properties of phonetics.
When McGurk stimuli was provided using fusions, responses were incorrect as to the sound made. For example, someone saying “ba” but appearing to say “ga” produced a perception that they were saying “ga.”
The McGurk effect occurs when our brains are attempting to give us its best guess as to what we are hearing. The audiovisual speech is confusing and contradictory, but one of them has greater influence on the brain, generating particular responses.
A University of Delaware study demonstrated that people integrate what they hear with what they see from a person talking when they perceive speech.
Since McGurk and MacDonald first introduced the concept, several studies have confirmed that visual and auditory information merges to form an integrated perception of speech.
We truly need both to operate because, with the exception of people who only read lips to understand what is being said, most people are extremely limited when it comes to using only their vision to "hear."
More for You:
NyRee Ausler is a writer from Seattle, Washington, and author of seven books. She covers lifestyle and entertainment and news, as well as navigating the workplace and social issues.