NHS England rushes to hide software over AI hacking fears

NHS England is hurriedly withdrawing all the software it has written from public view because of the perceived risk of hacking from cutting-edge artificial intelligence. Security experts say the move is unnecessary and counterproductive.

Software produced by the National Health Service has previously been made open-source and listed on GitHub because it is created with public money. This allows other organisations to build upon it and make better services more cheaply without duplicating effort.

But NHS England has issued new guidance to staff, which has been shared with New Scientist, that demands existing and future software be pulled from public view and kept behind closed doors. “All source code repositories must be private by default. Repositories must not be public unless there is an explicit and exceptional need, and public access has been formally approved,” says the new guidance. The deadline for making code private is 11 May.

Last month, an AI created by Anthropic called Mythos was widely reported to be capable of discovering flaws in virtually any software, potentially allowing hackers to break into systems running it.

NHS England’s guidance specifically points to Mythos as the cause for the new measures. “Public repositories materially increase the risk of unintended disclosure of source code, architectural decisions, configuration detail, and contextual information that may be exploited – particularly given rapid advancements in Al models capable of large-scale code ingestion, inference, and reasoning (e.g. developments such as the Mythos model),” it reads. “This red line establishes a default-closed posture for code while the organisation assesses the impact of these changes and ensures that any public publication of code is a deliberate, reviewed, and justified decision.”

However, the UK government-backed AI Security Institute (AISI) investigated Mythos and found it to be capable of attacking only “small, weakly defended and vulnerable enterprise systems”, concluding there was no indication that a really secure bit of software or network would be at risk.

The new measures go against the NHS service standard, which demands that staff make any software they produce open-source. “Public services are built with public money. So unless there’s a good reason not to, the code they’re based [on] should be made available for other people to reuse and build on. Open-source code can save teams [from] duplicating effort and help them build better services faster,” says the previous guidance.

Open-source software for public services also creates greater trust and transparency. For instance, if the code for the Horizon IT system that led the UK’s Post Office to pursue innocent people for alleged theft and fraud had been public, then the scandal might not have continued for years.

Terence Eden, who has extensive experience in the UK Civil Service working on opening access to public data, says the move makes no logical sense.

“Is it possible that Mythos will scan a repository and find a bug? Yes, 100 per cent likely. Is that going to be a bug that causes a security issue in a live NHS service somewhere? Almost certainly not,” says Eden. “I think it’s someone in NHS England buying into the hype that Mythos is going to cause the end of security as we know it and getting a bit panicked.”

Eden says open-source software is actually more secure because lots of people can check it for flaws, and most NHS software is not critically related to security in any case. Crucially, given that the code has been publicly available for years, it will continue to exist in various backups and downloads anyway.

“Shutting it down now is very much bolting the stable door after the horse has gone,” says Eden. “Myself and the people that I’ve spoken to within the NHS are just completely confused as to what this is trying to achieve.”

A spokesperson for NHS England said: “We are temporarily restricting access to some NHS England source code to further strengthen cyber security while we assess the impact of rapid developments in AI models. We will continue to publish source code where there is a clear need.”