An optimal transport and information geometric framework for infinite-dimensional Gaussian measures and Gaussian processes (II)

Divergences between probability distributions play a crucial role in many areas of probability theory, statistics, machine learning, and their applications. While a large part of the literature is focused on divergences between finite-dimensional distributions, there is a growing body of work on infinite-dimensional distances/divergences, which are motivated by applications in functional data analysis, Bayesian inverse problems, and functional Bayesian neural networks, among others. In this lecture, we present an overview of recent results on some of the most important divergences being studied, including the Kullback-Leibler, Renyi, and Geometric Jensen-Shannon divergences. We discuss the many challenges that arise in the infinite-dimensional setting, e.g. the lack of a natural reference measure such as the Lebesgue measure and the fact that many functions such as determinants and logarithm are only well-defined in specific settings. In particular, in the setting of Gaussian measures on infinite-dimensional Hilbert spaces, the closed form expressions for the above divergences are only generalizable to equivalent Gaussian measures. We present the resolution to the above challenges via the geometrical framework of positive definite unitized (or regularized) trace class and Hilbert-Schmidt operators, including the Alpha and Alpha-Beta Log-Determinant divergences. Using this framework and the methodology of reproducing kernel Hilbert spaces (RKHS), we furthermore obtain consistent finite-dimensional  approximations of the above divergences in the Gaussian process setting, with dimensional-independent sample complexities. The resulting numerical algorithms can be readily employed in practical applications. We shall also discuss the generalization of the above classical divergences above to the quantum setting, namely the Quantum Jensen-Shannon divergence between quantum states, defined in terms of the von Neumann and Tsallis entropies, from finite to infinite-dimensional settings.