**The complex-valued Neural Network is an extension of a (usual) real-valued neural network, whose input and
output signals and parameters such as weights and thresholds are all complex
numbers (the activation function is inevitably ****a complex-valued function****). **

**Neural Networks**** have been applied to various fields such as communication systems, image
processing and speech recognition, in which ****complex numbers**** are often used through the ****Fourier Transformation****. This indicates that ****complex-valued neural networks**** are useful. In addition, in the human brain, an action potential may have
different pulse patterns, and the distance between pulses may be different.
This suggests that introducing**** complex numbers**** representing ****phase**** and ****amplitude**** into neural networks is appropriate. **

**In these years the complex-valued neural networks expand the application
fields in image processing, computer vision, optoelectronic imaging, and
communication and so on. The potentially wide applicability yields new
aspects of theories required for novel or more effective functions and
mechanisms.
**

Merits of complex-valued neural networks |
（１） Representation of informationSince the input and output signals are supposed to be complex numbers (i.e.,
2 dimensions), the complex-valued neural networks can represent 2-dimensional information naturally,
needless to say complex-valued signals.（２） Characteristics of learning The learning speed of the complex-valued back-propagation learning algorithm
(called Complex-BP) for multi-layered complex-valued neural networks is 2 or 3 times faster than that of the real-valued one (called Real-BP). In addition, the required number of parameters such as the weights and the thresholds is only about the half of the real-valued case. An example of the application： （a） The authors applied the Complex-BP algorithm to the recognition and classification of epileptiform patterns in EEG, in particular, dealing with spike and eye-blink patterns. They reconfirmed the characteristics of learning described above. Reference：De Azevedo, F. M., Travessa, S. S. and Argoud F. I. M.., "The Investigation
of Complex Neural Network on Epileptiform Pattern Classification",
Proc. The 3rd European Medical and Biological Engineering Conference (EMBEC'05) , pp.2800-2804, 2005. An example of the improvement of the Complex-BP algorithm： （a） The authors proposed a modified error function by adding a term to the conventional error function in order to speed up the learning process. Reference： Chen, X., Tang, Z, Variappan, C., Li, S. and Okada, T., "A Modified Error Backpropagation Algorithm for Complex-valued Neural Networks", International Journal of Neural Systems, Vol.15, No.6, pp.435-443, 2005. |

Inherent properties of complex-valued neural networks |
（１） Ability to learn 2-dimensional affine transformationsThe complex-valued neural network can transform geometric figures, e.g. rotation, similarity transformation and
parallel displacement of straight lines, circles, etc..Examples of the application of the abiilty to learn 2-dimensional affine trnasformations：（a） Application to the estimation of optical flows in the computer visionReference： Watanabe, A. et al., "A Method to Interpret 3D Motion Using Neural
Networks", IEICE Trans. Fundamentals, Vol.E77-A, No.8, pp.1363-1370, 1994. （b） Application to the generation of fractal imagesReference： Miura, M. and Aiyoshi, E.., "Approximation and Designing of Fractal
Images by Complex Neural Networks", IEEJ Trans. EIS，Vol.１２３，No.８，pp.1465-1472, 2003 (in Japanese)．（２）Orthogonality of decision boundariesA decision boundary of a complex-valued neural network basically consists
of two hypersurfaces that intersect orthogonally, and divides a decision
region into four equal sections. Several problems that cannot be solved
with a single real-valued neuron, can be solved with a single complex-valued neuron using the orthogonal property. （３）Structure of critical points The critical points ( satisfying a certain condition) of the complex-valued neural network with one output neuron caused by
the hierarchical structure are all saddle points, not local minina, unlike the real-valued case where a critical point is a point at which
the derivative of the loss function is equal to zero. |

1. Kobayashi, M., Muramatsu, J. and Yamazaki, H.,

"High Dimensional Neural Network by Linear Connections of Matrix"，

Content: The authors discussed neural networks with high-dimensional parameters including complex-valued neural networks from the point of view of the vector representation.

(English paper translated from the above paper)

Masaki Kobayashi, Jun Muramatsu, and Haruaki Yamazaki,

''Construction of High-Dimensional Neural Networks by Linear Connections of Matrices'',

Electronics and Communications in Japan, Part 3, Vol.86, No.11, pp.38-45, 2003

2. Ogawa, T. and Kanada, H., "Complex-Valued Network Inversion for Solving Complex-Valued Inverse Problems"，

Content: A method for solving complex-valued inverse problems based on multi-layered complex-valued neural networks, called complex-valued network inversion, is proposed.

(English version of the above paper where some of the simulation are omitted)

Takehiko Ogawa and Hajime Kanada,

''Network Inversion for Complex-valuedNeural Networks”,

Proceeding of the IEEE International Symposium onSignal Processing and

Information Technology in Athens, pp.850-855 (2005).

Back to the home page of T.Nitta