Statistical Study of Artificial Neural Networks
I'm studying artificial neural networks
from the standpoint of multivariate data analysis.
In particular, I'm interested in automatic selection
of network structures and architectures
using statistical techniques.
Generative Models for Vision
Generative models are stochastic models about
the generation of given data from hidden causes.
Estimation algorithms for the parameters of the models
are regarded as unsupervised learning algorithms
for the internal representation of the data.
Using image data,
such models can learn visual internal representation
fit for the statistical characteristic of the images.
I am searching for the minimal generative model
to emulate the human vision system.
Mixture of Factor Analyzers
The Mixture of Factor Analyzers (MFA)
is one of unsupervised learning models and
a straightforward fusion of the mixture-of-Gaussian model
and the factor-analysis model.
MFA represents a hidden structure in data by
a set of local linear manifolds.
Since such a framework of hidden structures is very flexible and
expansive, it is expected to be the core of adaptive cognition systems.
Bayesian Self-Organizing Map
Self-Organizing Map (SOM)
is a unique neural network model
which has the ability to extract topological structures
hidden in environmental data.
Such discovered structures may be useful for the improvement of
systems for prediction, classification and control.
In reality, SOM is too flexible and fits to any data distributions
whichever topology it postulates.
I gave this over-flexible model a stiff property and
an ability to control this stiffness by itself
on a statistical principle.
This results in a convenient data-analysis tool without cumbersome trial-and-error tuning.
Demonstrations using java applets and GIF-animations
- A. Utsugi (2000) "Bayesian sampling and ensemble learning in generative topographic mapping",
Neural Processing Letters, vol. 12, no. 3, pp. 277-290.
- A. Utsugi (1998) "Density estimation by mixture models with smoothing priors",
Neural Computation, vol. 10, no. 8, pp. 2115-2135.
- A. Utsugi (1997) "Hyperparameter selection for self-organizing maps",
Neural Computation, vol. 9, no. 3, pp. 623-635.
(reprinted in "Self-Organizing Map Formation, Foundations of Neural Computation",
eds. K. Obermayer and T. J. Sejnowski (2001), MIT Press, pp.277-289.)
- A. Utsugi (1996) "Topology selection for self-organizing maps",
Network: Computation in Neural Systems, vol. 7, pp. 727-740.
- A. Utsugi (1994) "Lateral interaction in Bayesian self-organizing maps",
Trans. IEICE D-II, vol. J77-D-II, pp. 1329-1336. (in Japanese)
- A. Utsugi (1993) "A Bayesian model of topology-preserving map learning",
Trans. IEICE D-II, vol. J76-D-II, pp. 1232-1239. (in Japanese)
I studied mathematical models for various application areas
including human-computer interface, image sensing and
emotional decision making.
- M. Kitajima, A. Utsugi (1993)
"Validation of the fuzzy model of attraction emotions",
Japanese Journal of Fuzzy Theory and Systems, vol. 5, pp. 55-67.
- A. Utsugi, M. Ishikawa (1991)
"Construction of Inner Space Representation of Latticed Network Circuits by Learning",
Neural Networks, vol. 4, pp. 81-87.
- A. Utsugi, M. Ishikawa (1991)
"Learning of Linear Associative Mapping by Latticed Network Circuits",
Systems and Computers in Japan, vol. 22, pp. 56-65.