"Jeffreys' prior for layered neural networks"
In this paper, Jeffreys' prior for a neural network is
discussed in the framework of the Bayesian statistics.
For a good performance of generalization, the regularization methods which
reduce both cost function and regularization term are commonly used.
In the Bayesian statistics, the regularization term can be naturally derived from
prior distribution of parameters. Jeffreys' prior is known as a typical non-informativ
e objective prior. In the case of neural networks, however,
it is not easy to express Jeffreys' prior as a simple function of parameters.
In this paper, numerical analysis of Jeffreys' prior for neural networks
is given.
The approximation of Jeffreys' prior is given from a parameter transformation
getting to make Jeffreys' prior as a simple function.
Some learning techniques are also discussed as applications of these results.
|