Study by Alexandra Jesse and Michael Bartoli Suggests We Can Recognize Speakers by Dynamic Facial Movements Alone

alexandra jesseFindings from a new study by cognitive psychologist and speech scientist Alexandra Jesse and her linguistics undergraduate student Michael Bartoli found that adults can recognize unfamiliar people by not only their static facial features but by the dynamic ways they move their mouth and facial muscles while speaking.  

People commonly use the technique of lip-reading in addition to listening to help themselves comprehend speech. Jesse's laboratory, the Language, Intersensory Perception, & Speech (LIPS) Lab, explored how we can quickly learn the dynamic movements made by an individual's face during several conversations. The researchers wanted to show that just these personal motion "signatures" alone can be used to identify a person who is speaking. 

face with point-light displaysJesse's team created a way to isolate and record only the facial motion of speakers to test this theory. They created what they call "configuration-normalized point-light displays." To create these point-light displays, they glued 23 white paper dots on the faces of two different speakers and videotaped them as they spoke simple sentences. The researchers showed videos of these speakers to human subjects, allowing them to view only the dots seen moving against a black background, no sound and no facial details. 

Participants in the experiment underwent a training phase, watching each video before responding with one of two names to indicate who they thought they saw. Initially they had to guess, as they had no facial features to reference. Through feedback on their performance, they began to learn the subtle differences between the two speakers using just the point-light displays.  At a subsequent test phase, no feedback was given, and listeners saw videos of the speakers saying new sentences.

“Listeners learned to identify two speakers, and four speakers in another experiment, from visual dynamic information alone. Learning was evident already after very little exposure,” the authors report. Further, they say, listeners formed abstract representations of visual dynamic signatures that allowed them to recognize speakers even when seeing them speak a new sentence.

Read News Office Release

UMass Amherst News & Media Relations. (2018, April 18). UMass Amherst Study Suggests We Can Recognize Speakers Only from How Faces Move When Talking. http://www.umass.edu/newsoffice/article/umass-amherst-study-suggests-we-can