The connection between Bayesian estimation of a Gaussian random field
Reconstruction of a function from noisy data
is key in machine learning and is often formulated as a
regularized optimization problem
over an infinite-dimensional reproducing kernel Hilbert space (RKHS).
The solution suitably balances adherence to the observed data
and the corresponding RKHS norm.
When the data fit is measured using a quadratic loss, this
estimator has a known statistical interpretation.
Given the noisy measurements,
the RKHS estimate represents the posterior mean (minimum variance estimate)
of a Gaussian random field with covariance proportional
to the kernel associated with the RKHS.
In this paper, we provide a statistical interpretation
when more general losses are used,
such as absolute value, Vapnik or Huber.
Specifically, for any finite set of sampling locations
(that includes where the data were collected),
the maximum a posteriori estimate for the signal samples is given by the
RKHS estimate evaluated at the sampling locations.
This connection establishes a firm statistical foundation for
several stochastic approaches used
to estimate unknown regularization parameters.
To illustrate this, we develop a numerical scheme that implements
a Bayesian estimator with an absolute value loss.
This estimator is used to learn a function
from measurements contaminated by outliers.