Fairy Tales and Sleep; Cognitive Impairment in AI

TTHealthWatch is a weekly podcast from Texas Tech. In it, Elizabeth Tracey, director of electronic media for Johns Hopkins Medicine in Baltimore, and Rick Lange, MD, president of the Texas Tech University Health Sciences Center in El Paso, look at the top medical stories of the week.

This week’s topics include taxi driving and risk for Alzheimer’s, fairy tales and sleep, manual dexterity in hospital workers, and artificial intelligence (AI) and cognitive impairment.

Program notes:

0:52 Large language models and cognitive impairment

1:50 Radiologists still prevail

2:50 Think twice before using large language models

3:25 Alzheimer’s disease mortality among taxi and ambulance drivers

4:25 Neuroimaging showed hippocampal enhancement

5:25 Navigation apps

6:33 Fairy tales as good night stories

7:33 Sleep hygiene

8:10 Manual dexterity among hospital staff

9:10 Surgeons best at it

10:10 Surgeons also had more salty language

11:12 End

Transcript:

Elizabeth: If you want to lower your risk for Alzheimer’s disease, should you start driving a taxi?

Rick: Should you be reading fairy tales as good night stories?

Elizabeth: Who has got the best manual dexterity among people who work in hospitals?

Rick: And large language models that have cognitive impairment.

Elizabeth: That’s what we’re talking about this week on TTHealthWatch, your weekly look at the medical headlines from Texas Tech University Health Sciences Center in El Paso. I’m Elizabeth Tracey, a Baltimore-based medical journalist.

Rick: I’m Rick Lange, president of Texas Tech University Health Sciences Center and dean of the Paul L. Foster School of Medicine. Ho, ho, ho!

Elizabeth: It is our annual look at TheBMJ‘s tongue-in-cheek studies that they typically put out there, so I’m going to toss the ball to you. Oh, master of the pun, which of them would you like to start with?

Rick: Let’s talk about large language models and cognitive impairment. We are talking about things like ChatGPT, Claude 3.5 Sonnet, and Gemini versions 1 to 1.5 developed by different companies that all institute AI and large language models.

This is an interesting study done by a neurologist and here is why. When they release these large language models, they occasionally blunder on occasions. For example, they’ll cite journal articles that don’t even exist. But actually, when they looked at how they perform regarding functions of human physicians at qualifying exams around the globe, they have outperformed cardiologists in the European Core Cardiology exams, they have outperformed Israeli residents in the internal medicine board exams, Turkish surgeons in the thoracic surgery exams, and German gynecologists in the German obstetrics and gynecology exams. In fact, even to the dismay of the neurologist, they’ve done better than neurologists on neurology exams. However, in a few domains — for example, with regard to radiologists, The Royal College of Radiologists, the Taiwanese Family Medicine, the American Shoulder and Elbow Surgeons exam — the physicians still seem to have the upper hand. Yes, pun intended.

The neurologist, concerned that in fact the large language models may not be living up to snuff, decided to do a neurologic exam. This is called the Montreal Cognitive Assessment exam. They did additional tests as well: the Navon figure, the Cookie Theft picture, the Poppelreuter figure, and the Stroop test to examine the various neurologic functions of the large language models.

Here is what they discovered. Except for one, all of these scored below the norm, indicating they had cognitive impairment. Moreover, when they looked at the older ones versus the younger one, the older ones tended to perform worse, indicating to them that in fact they were developing worsening cognitive impairment as they got older, very similar to humans. Their comments to us? Well, they may not be better than many humans. Before you use a large language model to get you out of a jam, think twice.

Elizabeth: OK. Well, I haven’t been using too many of them. I found it interesting. It’s still, to me, kind of a curiosity. I guess I’m wondering is there sort of a quick and dirty cognitive assessment that can be done to a large language model to determine whether it’s experiencing decline.

Rick: It’s probably very difficult to do that either for older physicians or older large language models. I’m not sure that I have an answer, but I would say that I don’t think they are ready to replace physicians at this time. But stay tuned!

Elizabeth: OK. Let’s turn to the same issue, TheBMJ, and look at Alzheimer’s disease mortality among taxi and ambulance drivers.

This is actually quite a large study. What they looked at was the National Vital Statistics System in the United States and looked at occupations and death certificates in the Jan. 1, 2020 to December 2022 time period, all deceased adults aged 18 years and older. They also looked at 443 occupations. They looked at the percentage of deaths that were attributable to Alzheimer’s disease. About 9 million people who died during that time period with occupational information, just shy of 4% of those folks had Alzheimer’s disease listed as the cause of death on their death certificates.

What they found was that the taxi drivers and the ambulance drivers had the lowest rates among all of these occupations in having Alzheimer’s disease listed on their death certificates. They cite some previous work that showed that a neuroimaging study on taxi drivers in the U.K., specifically in London, showed that they had enhanced functional changes in their hippocampus in their brain. And so they are speculating that, gosh, maybe it’s the fact that you’ve got to do real-time integration of mapping in your head and accumulate additional information and respond to it quickly, and that that might somehow be protective.

Rick: What they used as a comparison group were other people that were involved in transportation. They used aircraft pilots, ship captains, and bus drivers. They did not have the same decrease in Alzheimer’s because these are individuals that they transport, but largely they do the same thing over and over, or sometimes they use computer-assisted navigation.

All right. Elizabeth, here is the question to you. My wife and I are in the car. We have different navigation skills. Hers are clearly better because she is giving me directions. Is her risk of Alzheimer’s less than mine?

Elizabeth: Isn’t that a good question? I got nothing on that. Gosh, since we all rely upon — speaking of large language models — our navigation apps, I’m not sure that those are really beneficial in terms of this issue of our ultimate risk of developing dementia of any type, whether that’s Alzheimer’s or something else.

One thing I did notice in here, and I’d like to draw your attention to it, is this table where they show that in terms of age, our ambulance drivers and our taxi drivers were the youngest folks among these different cohorts that had other occupations that also involved transportation. So I’m just wondering. Hmm, yes, and Alzheimer’s is something that develops as we age. Is there an issue with that? I know they attempted to correct for it, but I’m not all that persuaded by the correction.

Rick: Yep. Again, we have to take this tongue in cheek. If you were to look at this scientifically and say “Is it the fact that taxi drivers are less likely to die of Alzheimer’s because of their job or once they develop Alzheimer’s, they’re less likely to be taxi drivers?” With regard to your previous comment about my wife, and you called it a large language model, I’m not sure I’m going to refer to her as that the next time she’s giving me directions in the car.

Elizabeth: You’re a wise man and no doubt accounts for the success of your marriage. Let us move on.

Rick: Elizabeth, I teed this up as, “Should we be reading fairy tales as good night stories?” Fairy tales are meant to soothe the child to go to sleep. I actually have a different perspective from a healthcare provider and I actually think the fairy tales tell a lot about sleep disorders.

Let’s, for example, talk about Snow White. Now, for those that aren’t familiar, Snow White runs away from a wicked queen and she lives with seven little men named by Walt Disney as Doc, Grumpy, Happy, Sleepy, Bashful, Sneezy, and Dopey. These are dwarfs. One of the most well-known causes of being a dwarf is achondroplasia. It is an autosomal dominant feature, which is why all of them have it. By the way, it’s associated with obstructive sleep apnea, so it should be no surprise that these guys are not getting good sleep. Doc has a speech impediment and we know that sleep deprivation can cause that as well.

Using these fairy tales tells us a lot about sleep disorders. “Goldilocks and the Three Bears,” well, you remember that after sampling porridge and chairs and feeling tired, she goes to find a bed to sleep in. The first two beds are not comfortable. One is too soft, one is too hard, but then she finds just the right one. Obviously, a key part of achieving healthy sleep is optimizing our environmental conditions.

Finally, Peter Pan and Wendy. I’ll remind you that the kids have nightmares. Remember, the kids were separated from their parents. They have stress, separation anxiety, and sleep deprivation. Peter Pan is there to rescue them, but this causes nightmares and so the fairy tales tell us a lot about sleep disorders. I’m not sure they put the kids to sleep as well as they instruct them about the benefits of healthy sleep and the problems with unhealthy sleep.

Elizabeth: These life lessons as reflected in fairy tales. Of course, that is what they are all about anyway.

Finally, let’s turn to this last one, which is an assessment of the manual dexterity and composure under pressure of people in the different hospital staff roles using a buzz wire game. Now, I don’t know if you recall this game when I was a kid. Do you recall this game Operation? Where you had to insert these little tweezers into this cut out of a person and take out like their femur, or their humerus, or other body parts? It was the same kind of mechanism. If you touch the sides, it would go “bzzz.” Do you remember that?

Rick: I do. I played that as well. And I actually played this particular buzz wire game that they show where there is a wire — it looks like it’s convoluted, it has bends and stuff — and there’s a wand and you have to trace around it without ever touching the sides. That’s what was tested here.

Elizabeth: Exactly. They had 254 hospital staff members — 60 physicians, 64 surgeons, 69 nurses, and 61 non-clinical staff — and they asked them to get this wand all the way around the buzz wire within 5 minutes. The surgeons were the best at that — and I know that’s a news flash, what a shocker, right, for all of us — 84% of them could do it. Only 57% of the physicians, 54% of the nurses, and only 51% of the non-clinical staff.

Interestingly, surgeons experienced the highest rate of swearing during the game. The non-clinical staff showed the highest use of frustration noises, followed by nurses, surgeons, and physicians. They do admit that this was a single observer and there may have been some bias with regard to the identification of frustration noises while this game was going on, but they conclude that implementation of a surgical swear jar initiative might be considered if they are interested in future fundraising events.

Rick: Elizabeth, this was fun because they filmed them doing the game. They didn’t tell them what they were looking for. They just said, “Can you do this in less than 5 minutes?” As you said, the surgeons did the best. However, they had the — how do we put it? — the more salty language during this as well. The non-clinical staff were more likely to say “hmm” or “argh,” some other visual or auditory cue short of swearing, by the way.

Elizabeth: I think it’s worth noting that they used swearing. Their definition was any swear word not suitable for broadcast before the 9:00 p.m. watershed on U.K. television according to a publicly available list of offensive language published by Ofcom [Office of Communications]. Who knew?

Rick: Because they are not suitable for public use, we can’t say them on our podcast.

Elizabeth: On that note, that is a look at this week’s medical headlines from Texas Tech. Merry Christmas to everyone! I’m Elizabeth Tracey.

Rick: And I’m Rick Lange. Y’all listen up and make healthy choices during the holiday and in the new year. Merry Christmas!