Alexandra Jesse PhD, assistant professor at UMass Amherst, will present a talk titled Learning about speaker idiosyncrasies in visual speech

Abstract: Seeing a speaker typically improves speech perception, especially in adverse conditions. Audiovisual speech is more robustly recognized than auditory speech, since visual speech assists recognition by contributing information that is redundant and complementary to the information obtained from auditory speech. The realization of phonemes varies, however, across speakers, and listeners are sensitive to this variation in both auditory and visual speech during speech recognition. But listeners are also sensitive to consistency in articulation within a speaker. When an idiosyncratic articulation renders a sound ambiguous, listeners use available disambiguating information, such as lexical knowledge or visual speech information, to adjust the boundaries of their auditory phonetic categories to incorporate the speech sound into the intended category. This facilitates future recognition of the sound. For visual speech to best aid recognition, listeners likewise have to flexibly adjust their visual phonetic categories to speakers. In this talk, I will present work showing how lexical knowledge and auditory speech information can both assist the retuning of visual phonetic categories to speakers, but that at least the latter type of retuning seems to rely on attentional resources. Furthermore, listeners rapidly form identity representations of unfamiliar speakers’ facial motion signatures, which subserve talker recognition but may also aid speech perception.

For information on Dr Jesse, visit umass.edu/pbs/people/alexandra-jesse

All are welcome!