Ronald A. Fisher’s 1922 paper helped define key ideas of modern inference, including the likelihood function (how well a parameter value explains observed data). This work pushed statistical practice away from older “inverse probability” arguments and toward a distinct frequentist program centered on sampling behavior and estimation performance.
Andrey Kolmogorov’s 1933 foundations gave probability a rigorous measure-theoretic basis. This did not settle Bayesian-versus-frequentist disputes by itself, but it provided shared mathematical ground for both schools to build on.
Jerzy Neyman and Egon Pearson’s 1933 paper formalized hypothesis testing around long-run error rates (Type I and Type II errors) and test power. This became a central frequentist approach for “yes/no” testing decisions, distinct from Fisher’s significance-testing style.
Harold Jeffreys’ 1939 book developed a Bayesian approach to scientific inference, including tools for comparing hypotheses using what later became widely known as Bayes factors (evidence ratios). The work helped keep Bayesian methods alive during decades when frequentist testing was dominant in many applied fields.
The 1953 Metropolis et al. paper introduced a Monte Carlo algorithm (now called the Metropolis method) for sampling from complex probability distributions. This later became a key building block for Markov chain Monte Carlo (MCMC), which made Bayesian inference practical in many real problems.
Leonard J. Savage’s 1954 book argued that coherent choices under uncertainty can be represented using subjective probabilities and utilities. It strengthened the Bayesian “personal probability” tradition and clarified how Bayesian updating can guide decisions, not just parameter estimation.
Allan Birnbaum’s 1962 paper argued that combining the sufficiency principle (use all relevant data reduction) and conditionality principle (condition on the experiment actually performed) leads to the likelihood principle. This intensified foundational debate, because strict likelihood reasoning fits naturally with many Bayesian arguments and clashes with some common frequentist procedures.
The 1977 EM (expectation–maximization) algorithm provided a general way to compute maximum likelihood estimates with missing or latent (unobserved) data. EM strengthened frequentist likelihood-based modeling in complex settings, while also connecting to Bayesian ideas because many “missing data” problems can be approached by integrating over unknowns.
In 1984, Stuart and Donald Geman used Gibbs distributions and stochastic simulation for image restoration, helping bring “Gibbs sampling” style MCMC into wider attention. This was part of a larger shift where computation began to make Bayesian modeling feasible for high-dimensional problems.
The 1992 Gelman–Rubin paper introduced a widely used method for checking whether multiple MCMC runs have mixed well enough to trust posterior summaries. Practical diagnostics like this helped Bayesian methods move from theory into routine applied scientific work.
The 1995 Benjamini–Hochberg procedure introduced controlling the false discovery rate (FDR)—the expected proportion of false positives among the findings declared “discoveries.” This reframed frequentist multiple-testing practice in large-scale scientific studies, especially where thousands of hypotheses are tested at once.
In 1995, Kass and Raftery’s influential review helped standardize Bayes factors as a practical tool for model and hypothesis comparison. Their synthesis made Bayesian hypothesis testing more accessible to applied researchers and put Bayesian evidence measures into direct conversation with p-values and likelihood ratios.
E. T. Jaynes’ posthumously published 2003 book presented probability theory as an extension of logic for reasoning under uncertainty. It argued strongly for Bayesian updating and for “objective” methods of choosing priors in some settings, influencing scientists and engineers beyond statistics departments.
In 2016, the American Statistical Association released a statement cautioning against common misuses of p-values, especially treating them as a simple stamp of truth or importance. The statement reflected growing concern about reproducibility and helped reopen discussion of alternatives, including Bayesian methods and richer reporting practices.
Stan’s probabilistic programming language supports efficient Bayesian inference using advanced MCMC (including Hamiltonian Monte Carlo variants). Tools like Stan lowered the cost of fitting complex Bayesian models, accelerating Bayesian adoption in fields such as social science, biology, and medicine.
A 2019 special issue in The American Statistician urged the community to stop using “statistical significance” as a bright-line rule. This moment did not end frequentist methods, but it clarified a new end state: many scientists now treat inference as a toolbox where Bayesian and frequentist approaches are chosen (and reported) with more emphasis on transparency, uncertainty, and context.
Bayesian Methods versus Frequentist Approaches in Scientific Inference (1920s–present)