How Patients Speak May Signal Cognitive Impairment

Short segments of conversations between primary care clinicians and patients contained signals that helped detect undiagnosed cognitive impairment, an acoustic analysis suggested.

A machine learning model trained on acoustic features from recordings of primary care visits achieved a sensitivity of 68.2%, specificity of 63.6%, and positive predictive value of 30.4% for identifying cognitive impairment, reported Joseph Colonel, PhD, of the Icahn School of Medicine at Mount Sinai in New York City, and co-authors.

The model had an area under the receiver operating characteristic curve (AUROC) of 0.733 and a maximum F1 score (F_max) of 0.502, Colonel and colleagues wrote in JAMA Neurology.

AUROC and F_max values were similar in a validation cohort. Measures of pitch, timing, and speech variability were key predictors of cognitive impairment.

The project was designed to learn how machine learning models could be used in primary care to screen for cognitive impairment, “as it is frequently undiagnosed or underdiagnosed,” Colonel said in a JAMA Network podcast interview. “Are there patterns in a conversation between a patient and their physician that might indicate that there are some problems with cognition?”

Of five approaches tested in the study, the best-performing model was one trained on prosodic features — acoustic elements like intonation, stress, and tempo.

“Those relate to how someone talks,” Colonel said. “How does the pitch of their voice change? How does the volume of their voice change? How quickly are they speaking?”

The study showed that “the more quickly someone spoke, the more that was related to healthy cognition,” he noted. “Things related to pause duration were more positively associated with cognitive impairment.”

Earlier research assessed whether patients’ vocal responses to questions on tests could predict dementia, or whether models could distinguish normal versus impaired cognition by voice alone. In a small study in Japan, a predictive model identified vocal features of dementia from speech patterns in phone conversations. Most previous studies were based on structured tasks, not unstructured conversations, Colonel pointed out.

Primary care clinicians are positioned to detect signs of decline in their patients but only 8% of expected mild cognitive impairment cases are diagnosed in primary care settings, observed Gabriela Meade, PhD, and Hugo Botha, MBChB, both of the Mayo Clinic in Rochester, Minnesota.

“This gap is driven by numerous factors, including restricted time, the number of co-occurring needs, and the perception that existing assessment tools are unhelpful,” Meade and Botha wrote in an accompanying editorial. “One potential solution is to embed cognitive screening into existing clinical workflows; researchers have increasingly turned to speech as a potential passive screening method.”

If machine learning models were further developed, “this approach could meaningfully change how and when cognitive decline is detected,” they added.

From August 2020 through December 2021, Colonel and co-authors studied 787 older adult patients in New York with no history of dementia or mild cognitive impairment, and 179 similar patients in Chicago as a validation cohort.

The mean age in the study was 67.2 years and 55% of participants were women. Overall, 35% of participants were Black, 33% were white, 22% were Latinx, and 10% were of another race.

Patients were invited to attend an in-person interview immediately after their primary care appointment. The researchers used audio recordings collected during these clinical visits to train machine learning classifiers.

The study’s primary outcome was cognitive impairment, defined as a Montreal Cognitive Assessment (MoCA) score at least 1 standard deviation below age- and education-adjusted norms. Based on MoCA scores, the overall rate of undiagnosed cognitive impairment was 21%.

The classifiers performed better when trained on recordings that captured both patient and physician speech rather than patient speech alone. The top-performing model used acoustic features derived from Whisper.

The analysis looked only at the acoustic properties of the primary care conversations, not the content, the researchers acknowledged. Future work should validate the findings in larger, more diverse populations and should incorporate electronic health record data, they said.