Are Bayesian Statistics Coming to a Clinical Trial Near You?

The FDA’s move to incorporate Bayesian statistical methods into clinical trials of drugs and biologics garnered special treatment in JAMA this week.

JAMA published three perspectives — two welcoming the agency’s draft guidance, published in January, and one urging caution.

Among the authors of the piece urging caution is Thomas Fleming, PhD, of the University of Washington in Seattle, who has long been a key contributor to standard statistical methods for clinical trials, also known as the “frequentist” approach.

The other two papers feature authors who have supported Bayesian methods, including Andrew Gelman, PhD, of Columbia University in New York City, and J. Jack Lee, PhD, of the University of Texas MD Anderson Cancer Center in Houston.

So what’s the difference between frequentist and Bayesian methods? While the frequentist approach assumes a drug’s effect is a fixed, unknown constant, and begins with a null hypothesis — that the drug doesn’t work — the Bayesian approach assumes that the drug’s effect is a probability distribution.

Bayesian analysis incorporates prior evidence — referred to as “priors” — and combines that with new data to update beliefs, called the “posterior,” Gelman and Lee told MedPage Today.

“One problem in research is that the larger effects are statistically detected and the smaller ones are below the limit of detection,” Gelman told MedPage Today. “As a result, the things that get reported tend to be overestimates. With a Bayesian approach, the large, positive effects tend to be pulled a bit towards zero. … But then the estimates that are small, you still allow there to be a certain probability that something is effective. You don’t just say it doesn’t work because it’s not statistically significant at some threshold.”

Lee uses the analogy of weather reports. You check the weather at night and there’s a 40% chance of rain in the morning. If you look out your window the next morning and the sky is dark, you might think the chances have increased to 80% and you carry an umbrella. If the skies are clear, you may update the estimate to a 10% chance of rain and leave the house without one, he said.

But therein lies the rub; frequentists believe that the “priors” necessary for Bayesian analysis introduce bias — and that their standard methods control better for type I error, or false positives.

Fleming and colleagues noted that use of Bayesian methods in late-stage or confirmatory trials thus far has been limited to supplementary analyses, “given recognition that Bayesian methods could be implemented in a manner that would compromise evidentiary and integrity standards and the reliability of results.”

However, Gelman said “people of all statistical persuasions accept the idea of using prior information in the design of a study whether it’s Bayesian or not. You design a study so that you have a shot at detecting a realistic effect size. … In that way, designs are already Bayesian, even if they are not formally.”

And Lee noted that adaptive clinical trial designs in oncology already exist, and these are essentially a Bayesian approach. They work well because “we learn as we go,” Lee said.

Biostatisticians are also comfortable using Bayesian approaches when it comes to things like positive and negative predictive value for diagnostic tests. Even if a test has a 95% sensitivity and a 95% specificity — no such test exists, Lee noted — you still need the “prior” information of disease prevalence to calculate positive and negative predictive value, which is the “posterior,” he said.

Lee also pointed out that the Fleming paper acknowledges Bayesian approaches can help in settings such as pediatrics or rare disease when the number of cases would be too low to have a sufficiently powered randomized clinical trial.

Nonetheless, both Lee and Gelman acknowledged that it’s critical for the “prior” information in a Bayesian analysis to be of high quality.

Lee also noted that it would take some time for physicians to get comfortable with the new terminology. While they’re already well-versed in the frequentist’s P-value and confidence interval, they’d have to get used to the Bayesian approach’s prior, posterior, and credible interval.

FDA’s draft guidance on Bayesian approaches for clinical trials in drugs and biologics has been many years in the making, Lee added. While the agency’s centers for drugs and biologics have been slow to adopt this approach, the Center for Devices and Radiological Health adopted guidance for use of Bayesian statistical models in device trials in 2010.

He also noted that the Bayesian approach actually predates the frequentist model. Bayes’ Theorem was named for Thomas Bayes, an 18th-century clergyman and amateur mathematician whose work was published posthumously in 1763, 2 years after his death. However, his work didn’t come into fashion until the 1980s as computing power and algorithms improved, Lee said.

In the meantime, frequentist methods, driven largely by the work of British statistician Sir Ronald Aylmer Fisher in the 1920s, became the prominent model in clinical trials and medical science, in part because it was also computationally easier to perform, Lee said.

The Fleming paper — which counts as a co-author an expert from the clinical trials research and statistics branch at the National Institute of Allergy and Infectious Diseases — takes issue with the January FDA press release announcing the draft guidance.

While the release framed the guidance as “modernizing statistical methods,” Fleming and colleagues argued that the “currently used methods that preserve objectivity and benefits of randomization are not outdated, old school, or closed-minded.”

“They are principled, based on scientific fundamentals for protecting integrity and ensuring robustness of scientific conclusions,” they wrote. “The commitment to achieving these objectives should be protected, not replaced.”