This paper should win a prize for its refusal to make any big claims

Feedback is New Scientist’s popular sideways look at the latest science and technology news. You can submit items you believe may amuse readers to Feedback by emailing [email protected]

Speaking our truth

The experienced science journalist soon learns to skim over certain sections of scientific papers: specifically, the sentences stating that the research represents “a significant advance” and “expands our understanding”. Not because they’re incorrect, but because literally any study that achieves anything at all can make these claims, and academics are incentivised (as we all are) to amplify the impact of their work.

Except for the times when they don’t bother. Via a long sequence of events that started with reporter Matthew Sparkes and went via the social network Bluesky, Feedback discovered a paper on the arXiv preprint server from 2018 that should win a prize for its absolute refusal to make any big claims.

In it, researchers Joseph Redmon and Ali Farhadi described their latest iteration of YOLO: one of those AI systems that can be trained to recognise objects in images. YOLO can beat those CAPTCHA tests that ask you to click all the squares that contain bicycles, and it has been used to spot smuggling ships. All of which is quite impressive/alarming (delete as appropriate), but by 2018 the pair were evidently coasting.

It starts with the paper’s title: “YOLOv3: An incremental improvement”. The short summary continues the trend by claiming: “We made a bunch of little design changes to make it better.” The main text begins: “Sometimes you just kinda phone it in for a year, you know? I didn’t do a whole lot of research this year. Spent a lot of time on Twitter.” That last line certainly dates the paper.

The authors go on to explain that they “mostly took good ideas from other people” to improve YOLO. They describe this in some detail, after first admitting that the tweaks are “honestly, nothing like super interesting, just a bunch of small changes that make it better”.

Then we get to section 4, which is titled “Things we tried that didn’t work”. Feedback thinks this should be included in all scientific papers as a matter of course. It would save other researchers so much time.

The authors confess that they have only described “the stuff we can remember”, but they do recall that they tried adding something called “focal loss”, and that it made the model less accurate. “YOLOv3 may already be robust to the problem focal loss is trying to solve,” they say, “because it has separate objectness predictions and conditional class predictions. Thus for most examples there is no loss from the class predictions? Or something? We aren’t totally sure.”

Feedback can’t quite believe we missed this in 2018, or when it was picked up on the aggregator site Reddit in 2024. But we are grateful to sociologist Per Engzell, who said on Bluesky that “Limitations sections are where academics practice radical honesty for exactly one paragraph”, and to data scientist Johan Ugander, who replied that the YOLOv3 paper should get an award for “most honest paper“.

Surely, someone must know of an academic being even more disarmingly honest about how little they have accomplished. Emails to the usual address.

A long-lived bit

“I know you are avoiding nominative determinism,” writes Clare Boyes, incorrectly, “but couldn’t resist sending you this one which came in an email today from the British Wildlife Newsletter.” It was a book called Tree Hunting: 1,000 trees to find in Britain and Ireland’s towns and cities by Paul Wood.

Likewise, Robert Masta points out that our recent special issue on “how to live to 100” (TL;DR don’t die) featured a longevity researcher named Paul Lazarus.

Sleep on this

Back in the mists of time (July), Feedback wrote about receiving a press release that staunchly defended the environmental sustainability of avocados, only to discover that it came from the World Avocado Organisation. We concluded that these people might be right or might be wrong, but, either way, they may have been operating in an incentive structure.

We haven’t heard anything further from the avocadomongers, but we did get a series of press releases about the importance of sleep. “Can’t find a solution? Science confirms that sleeping on it really does solve problems” announced the first. It went on to share “a fascinating new research” explaining that “the old advice to ‘sleep on it’ might actually be one of the smartest problem-solving tools we have”.

This is because the brain continues processing memories and forming new connections while we sleep, it explains, sometimes generating new insights by fusing new and old ideas. There’s talk of “memory consolidation”, “the prefrontal cortex (the brain’s inner critic)” and “associative thinking”.

A follow-up email went further, with a dramatic and grammatical title: “New study shows rising youth deaths and could worsen if sleep deprivation persists, experts warn”. The press release linked poor sleep to chronic health conditions. It also featured a quote from a “Certified Sleep Coach”, which may well be a real thing, but in our addled mind it generated an image of a sweaty man in a tracksuit, blowing a whistle and yelling at us to “give me seven [hours]!” Still, the message was clear: sleep good.

Possibly the foreshadowing at the start gave it away, but in case you hadn’t guessed, both emails were sent on behalf of Amerisleep, which is, of course, a supplier of mattresses.

Got a story for Feedback?

You can send stories to Feedback by email at [email protected]. Please include your home address. This week’s and past Feedbacks can be seen on our website.