Interpolation Models with Multiple Hyperparameters
David J C MacKay and Ryo Takeuchi
A traditional interpolation model is characterized by the
choice of regularizer applied to the interpolant, and the choice of
noise model. Typically, the regularizer has a single
regularization constant \alpha, and the noise
model has a single parameter beta. The ratio
\alpha/beta alone is responsible for determining globally
all these attributes of the interpolant: its
`complexity', `flexibility', `smoothness', `characteristic
scale length', and `characteristic amplitude'. We suggest
that interpolation models should be able to capture more than
just one flavour of simplicity and complexity. We describe
Bayesian models in which the interpolant has a smoothness
that varies spatially. We
emphasize the importance, in practical
implementation, of the concept of `conditional convexity' when
designing models with many hyperparameters.
We apply the new models to the interpolation of neuronal spike data
and demonstrate a substantial improvement in generalization error.
postscript (Cambridge UK).
postscript (Canada mirror).
David MacKay's:
home page,
publications.
bibtex file.
Canadian mirrors:
home page,
publications.
bibtex file.