Kurland Developing AI Tool for Measuring Communication Ability in People with Language Impairments

Content

Jacquie Kurland

Jacquie Kurland, associate professor of speech, language and hearing sciences, will use a three-year, $1.06 million funding award from the Patient-Centered Outcomes Research Institute (PCORI) to develop and test an AI tool that measures communication success in people with aphasia, a common language impairment following stroke and other left-hemisphere brain damage.

Aphasia can affect one’s ability to speak, understand spoken language, read and write. Often people with aphasia can communicate better in ways other than talking, says Kurland.

“This research is important because people with aphasia want to communicate better, and our methods of treatment and intervention have been moving toward this kind of real-world life participation approach,” Kurland says. 

But currently there are no clinically convenient tools for assessing patients’ communication ability due both to a lack of diverse, real-world scenario tests and the fact that assessing connected speech in aphasia is time and resource intensive. 

Those are the challenges the PCORI award will allow Kurland, along with a multidisciplinary team of UMass Amherst co-investigators, to address. Her collaborators are Manning College of Information and Computer Sciences associate professor Brendan O’Connor, director of the Statistical Social Language Analysis (SLANG) Lab; statistician Anna Liu, professor of mathematics; and health services researcher Thomas Mackie, professor of health policy and management. 

Ultimately, the researchers aim to deliver an open-source large language model (LLM), able to analyze story retelling tasks, that could become the first standard tool for measuring connected discourse in people with aphasia. Moreover, it will create opportunities for clinical trials comparing the effectiveness of treatments that target life participation approaches to improving communication among people of all ages with a variety of communication impairments.

“If we’re successful in developing this tool, it can be used by all kinds of people with communication disorders,” Kurland says. “People who’ve had traumatic brain injuries, right hemisphere stroke, dementia and other symptoms of neurodegenerative disease. Even children with learning or communication impairments.”

Clinicians still rely on outdated standardized tests to gauge the communication of people with aphasia. Typically, they ask the patient to discuss a decades-old picture of a picnic scene featuring gender stereotypes from the 1950s. “Nobody can relate to these pictures,” Kurland says.

In addition, these standardized tests don’t measure the ability of a person with aphasia to communicate with another person in a real-world, meaningful situation. With the PCORI award, Kurland and team are continuing work to develop a tool that asks patients to retell a short story they have viewed, such as a Brief But Spectacular series from PBS and StoryCorps clips played by NPR. “These are autobiographical stories of people from different ages, racial and ethnic backgrounds, socioeconomic status, etc., so that people with aphasia, who are increasingly diverse, can relate,” Kurland says.

The patients retell the story they have viewed to the best of their ability. Then they engage with a conversation partner to co-construct the story. Finally, the partner, who was new to the story prior to the conversation, retells the story. Then the LLM compares a transcript of the retold story to the original narration and determines how well the patient has communicated the main concepts, which is the gold standard for analyzing connected discourse. This kind of discourse analysis is very time-consuming and laborious even for a trained person to do. Kurland and colleagues’ preliminary research has shown that the LLMs can rate story retells as accurately as humans. 

The researchers also will conduct surveys and focus groups with stakeholders, including clinicians, researchers, people with aphasia and their family members and care partners, to assess how well the story retell program using automated discourse analysis tools might function in clinical practice. The study will also assess the LLM’s ability to generate main concepts for novel, personalized stories.

“This work could lead to a paradigm shift in clinicians’ and clinical researchers’ ability to assess real-world communication improvements in patients across a broad swath of populations with speech, language and communication impairments,” Kurland says.

PCORI is a nonprofit organization with a mission to fund research designed to provide patients, their caregivers and clinicians with the evidence-based information needed to make better-informed healthcare decisions.

Kurland’s award has been approved pending completion of a business and programmatic review by PCORI and issuance of a formal award contract.