I Served on Grant Review Panels for 12 Years. Let’s Not Romanticize NIH Peer Review.

For more than a dozen years, I served on federal scientific review panels. My colleagues were serious, conscientious, and deeply knowledgeable in their fields. But the process was also far less mechanical, objective, and insulated from judgment than many public defenders of peer review suggest.

That matters now because the Trump administration’s Office of Management and Budget (OMB) has proposed having senior political appointees conduct a “pre-issuance review” of every discretionary NIH grant. The proposal indeed raises serious concerns. It would create a standing mechanism for political officials to second-guess scientific awards before they are issued. Once built, such a mechanism would not belong only to this administration. It would be available to every future administration, with whatever priorities, orthodoxies, enemies, or enthusiasms it brings to power.

But opposing political review does not mean we should pretend the current system is pristine. That is where the editors of the New England Journal of Medicine (NEJM), in their recent criticism of the proposal, missed an opportunity. They are right to worry about political control of science. But they leaned on a too-comforting account of how scientific funding works now.

The public story of NIH review is familiar. Objective experts, shielded from politics, score each proposal on its scientific merits. The best applications rise to the top. Funding follows the science. That story contains some truth, but it is incomplete.

Panel members review applications, discuss them, score them, and send them on. In my years on study section, we were not told which applications were ultimately funded. That is because the priority score is only one input to the funding decision. After scientific review, applications move through advisory council review, institute staff, program officers, and institute leadership. NIH institutes make funding decisions in light of programmatic priorities, portfolio balance, career stage, and institutional goals. Some institutes also use “select pay” to fund applications below the payline because they advance priorities the institute values.

A measure of political discretion has lived inside this system as long as it has existed. The NEJM editors surely know this; the cleaner story simply serves their argument better.

Their second assumption fares no better: that expert peer review reliably sorts the worthy from the unworthy. A typical application is read closely by a small number of assigned reviewers, commonly three. The rest of the panel may participate in the discussion, but most have not read the proposal with the same care and often defer to the assigned reviewers’ judgments.

Which reviewers are assigned to a proposal can therefore matter enormously. Frequently, reviewers are generous, thoughtful, and perceptive. But sometimes they miss the point. Sometimes they impose their own methodological preferences. Sometimes they are simply not the right reviewers for an application. For a senior investigator, a bad draw may be a setback. For a junior investigator, it can be catastrophic.

Evidence supports what many applicants suspect: the review process is not entirely objective. In one study, researchers had many reviewers independently evaluate the same already-funded NIH grants. Agreement among reviewers was extremely low. That does not mean reviewers were careless or corrupt. It means the margin between excellent and merely very good applications is difficult to judge, and the process contains much more randomness than the official language of merit review tends to admit.

A better response to the OMB proposal would start with humility about what peer review can and cannot do. To his credit, NIH Director Jay Bhattacharya, MD, PhD, has previously acknowledged some of these weaknesses. He has called for a more unified funding strategy in which institutes transparently weigh scientific merit alongside mission priorities, program balance, and workforce needs, rather than treating the priority score as the whole story. And he has pressed NIH to reward what the “publish or perish” culture neglects: replication, reproducibility, and negative findings.

But political pre-issuance review is not the right remedy. It mistakes the existence of discretion in NIH funding for a reason to formalize political control over it. The problem with peer review is not that scientists have too much insulation from elected officials. It is that the system too often hides contestable judgments behind the language of scientific precision. And in the end, the OMB proposal would only serve to make the grant review process more opaque for everyone.

The trouble with hyperbole is that it crowds out the argument worth having. By invoking Trofim Lysenko, the Soviet agronomist whose state-imposed theories devastated Soviet biology, NEJM avoided the reform case it was especially well positioned to make: that American science already has real problems of hierarchy, risk aversion, incumbent advantage, and opaque decision-making. The choice is not between pristine science and Soviet pseudoscience. It is between repairing a flawed but indispensable system and replacing it with one that is more openly vulnerable to political direction by this administration and the next.

The worry is not that a latter-day Lysenko will get funded. It is another generation of promising young scientists losing grants while Washington and the academy argue over the wrong question. The scandal is not merely that politics may corrupt science. It is that a system already too deferential to insiders may now become too deferential to politicians.

Please enable JavaScript to view the comments powered by Disqus.