Introduction to statistical decision theory and Stein’s paradox

Statistical decision theory is a general framework for discussing optimality of statistical procedures such as estimation, testing and prediction. In 1956, Charles Stein found a counter-intuitive phenomenon in estimation of the mean parameter of a multivariate normal distribution. He showed that a ``shrinkage estimator” attains better estimation accuracy (smaller mean-squared error) than the maximum likelihood estimator when the dimension is greater than or equal to three. This phenomenon is related to several mathematical fields such as Markov processes and potential theory. The idea of shrinkage estimation has been employed in many statistical methods such as regularization, empirical Bayes and model selection. In this talk, I will introduce the statistical decision theory and illustrate Stein’s paradox.