The dangers of so-called AI experts believing their own hype

Demis Hassabis, CEO of Google DeepMind and a Nobel prizewinner for his role in developing the AlphaFold AI algorithm for predicting protein structures, made an astonishing claim on the 60 Minutes show in April. With the help of AI like AlphaFold, he said, the end of all disease is within reach, “maybe within the next decade or so”. With that, the interview moved on.

To those actually working on drug development and curing disease, this claim is laughable. According to medicinal chemist Derek Lowe, who has worked for decades on drug discovery, Hassabis’s statements “make me want to spend some time staring silently out the window, mouthing unintelligible words to myself”. But you don’t need to be an expert to recognise the hyperbole: the idea that all disease will be ended in around a decade is absurd.

Some have suggested that Hassabis’s remarks are just another example of tech leaders overpromising, perhaps to attract investors and funding. Isn’t this just like Elon Musk making silly forecasts about Martian colonies, or OpenAI’s Sam Altman claiming that artificial general intelligence (AGI) is just around the corner? But while that cynical view may have some validity, it lets these experts off the hook and underestimates the problem.

It is one thing when seeming authorities make grand claims outside their area of expertise (see Stephen Hawking on AI, aliens and space travel). But it might appear as if Hassabis is staying in his lane here. His Nobel citation mentions new pharmaceuticals as a potential benefit of AlphaFold’s predictions, and the algorithm’s release was accompanied by endless media headlines about revolutionising drug discovery.

Likewise, when his fellow 2024 Nobel laureate Geoffrey Hinton, formerly an AI adviser with Google, claimed that the large language models (LLMs) he helped create work in a way that resembles human learning, he seemed to be speaking from deep knowledge. So never mind the cries of protest from those researching human cognition – and, in some cases, on AI too.

What such instances seem to reveal is that, weirdly, some of these AI experts appear to mirror their products: they are able to produce remarkable results while having an understanding of them that is, at best, skin deep and brittle.

Here is another example: Daniel Kokotajlo, a researcher who quit OpenAI over concerns about its work towards AGI and is now executive director of the AI Futures Project in California, has said: “We’re catching our AIs lying, and we’re pretty sure they knew that the thing they were saying was false.” His anthropomorphic language of knowledge, intentions and deceit shows Kokotajlo has lost sight of what LLMs really are.

The dangers of supposing these experts know best are exemplified in Hinton’s comment in 2016 that, thanks to AI, “people should stop training radiologists now”. Luckily, experts in radiology didn’t believe him, although some suspect a link between his remark and growing concerns from medical students about job prospects in radiology. Hinton has since revised that claim – but imagine how much more force it would have had if he had already been given the Nobel. The same applies to Hassabis’s comments on disease: the idea that AI will do the heavy lifting could engender complacency, when we need the exact opposite, both scientifically and politically.

These “expert” prophets tend to get very little pushback from the media, and I can personally attest that even some smart scientists believe them. Many government leaders also give the impression they have swallowed the hype of tech CEOs and Silicon Valley gurus. But I recommend we start treating their pronouncements like those of LLMs themselves, meeting their superficial confidence with scepticism until fact-checked.

Philip Ball is a science writer based in London. His latest book is How Life Works