Use advanced Kohonen network to achieve fast learning machine vision
Fast Learning Computer Vision Using Advanced Kohonen Network Network NetWork
Liu Li
http://www.aivisoft.net/
Geo.cra [at] gmail [dot] com
[PDF format]
Summary: In this article, we describe an Advanced Kohonen network, and use this type of network to implement fast learning machine vision (LCV) that can learn multiple objects.
Keywords: kohonen, neural network, machine vision, machine learning
I. Introduction
As we all know, the Kohonen network is a representative of non-supervised learning networks, which has achieved great achievements in data mining. The greater advantage of the Kohonen network is that the mathematical algorithm without the bionics, such as the reverse propagation algorithm, in biology, Kohonen network mechanism has got an agreement, so the Kohonen network has a strong bionic character. However, due to the complexity of image data and some defects in the Kohonen network itself, the Kohonen network is currently not achieved in the field of machine visual fields.
Second, Kohonen Network Introduction
As a competitive mechanism, the Kohonen network has a non-supervised learning model. In many books, we are only briefly introduced, please refer to other books.
[3].
Kohonen Network is designed based on Winner-Takes-All principles, a total of two layers: one layer is an input neuron, and the other layer is a competitive neuron.
The main principle of the Kohonen network is by calculating the error E = σ (D [i] -w [i]), the neuron of the error is minimized as a winner, and the neurons in the radius r use W = W a * (dw) update the weight. So the connection weight record in the Kohonen network is not a connection intensity, but an object characteristics.
Third, Advanced Kohonen Design
In order to achieve NS1 machine vision (an important feature of NS1 machine vision is to automatically learn to identify), we have repeatedly considered, the Kohonen network has no need to reconstruct the network due to its unparalleled non-supervised learning. I have been favored. Using the Kohonen network to achieve machine vision, you have to consider the defect of Kohonen network. The Kohonen network is an important drawback in practice. This network is implemented without supervision. Although it can achieve good clustering effect, it is unsuccessful as a classifier because of Winner in the network. The weight is characterized by the cluster, and sufficient information that can be separated from other objects cannot be provided. Therefore, if a Kohonen network is simply used, it is difficult to achieve a sufficiently low misconduct.
In order to overcome this lack, we have established an environment model. The main guiding ideology of establishing an environmental model is to use as much environment model to make punitive learning, so that the Kohonen network provides adequate separation margin.
Currently, there are teachers in Kohonen
[5] The penalty method used is a method of decreasing the weight, that is, to take a negative value of the A-anegular A * (D-W).
However, this method also reduces the clustering effect while increasing the separation margin, is not a practical method. To this end, we use a so-called induction penalty method in the Advanced Kohonen network, which uses the ATTRACTIVE-CORRECT method. The idea of this method is to map the environmental model to other neurons, and the winning neurons of the objects are different from the winning neurons of the environment model, which greatly reduces the miscarriage.
Our research also shows that there are often similar methods in the process of human learning. In our experiments in humans, it is often told that the memory effect of an object of the experimenter is better than telling the memory of an object. However, since the induced punitive method is used, a portion of the neuron is used for the mapping of the environment model, so the number of neurons does not represent the maximum number of learning objects. During practice, we build 20 * 20 Advanced Kohonen networks can learn about 20 to 80 items (of course, this result is also related to the contents of the following) If the learning object continues to increase, misconditionation rate Will greatly improve. Therefore, if you want to achieve a practical effect, the network containing> 400 neurons is necessary, but due to the limitations of computer operation speed, each increase double neuron, the calculation cost is doubled, which is very disadvantageous.
Different from ordinary neural networks (ie, N-laying-based networks that use reverse propagation algorithm training), Advanced Kohonen network implements LCV does not need to use all training data to be restroom each time, only need to learn more items and environmental data. Mixed training can reach the requirements of the so-called incremental learning (Add-Learn). In this way, the Kohonen network can achieve a quick requirement at time.
Once the Ordinary Neural Network is established, it is necessary to train all training data, otherwise the network will have a strange weight variation, so that the cognitive performance of the previously learned items decreases. Therefore, the more the number of items learned in ordinary neural networks, the more slow learning speed, the more learning the era of online convergence, and the number of learning repetitions).
Comparison, it is easy to discover the Kohonen network also there is a value variant, and it seems more dangerous. An extreme example is that if the winning neuron of the new item is happening is the winning neuron of the previously learned object, there will be a forgotten phenomenon. After the study is over, the network will forget the previously learned objects. In order to overcome this shortcomings, in the Advanced Kohonen network, we will set the winning neurons of the objects that have been learned to be unavailable. That is, when learning, these neurons do not allow activation (neuronal fake) to ensure that memory is not lost. This is why the number of Advanced Kohonen network learning items containing 400 neurons is so small.
However, ordinary neural networks cannot achieve incremental learning in this way. Ordinary neural network If you want to make the neuronal macrosis, you can't make the output neurons or input neurons, because once the two types of neurons are killed, they can not be reversely propagated, so they can only make hidden layers The neuron is fake. However, after the hidden neuronus, the reverse propagation algorithm cannot use the neuron to continue to perform error propagation, indirectly reduces the number of corrected connections, improves the misthness rate, and is closely related to the generalization of hidden neurons and networks. Due to the limited network generalization performance, the continuous reduction in the number of hidden neurons cannot be predicted to reduce the specific effects of generic performance. However, it is foreseen that a death of a hidden neuron will change the structure of the entire network, resulting in an impact on the activity of the network. Moreover, it is more important that it is impossible to determine which hidden neurons should die in incremental learning, because it is unable to learn which hidden neurons are contributing to identifying those learning.
In summary, ordinary neural network is difficult to achieve incremental learning. However, for the Advanced Kohonen network, it is easy to achieve rapid incremental learning function due to the adoption of neuronal hypotransmission, and it does not affect the generalization of the network.
Three, practice
Here is an introduction to the LCV using the Advanced Kohonen network. The design of the network structure is as follows:
We use a grayscale image of 48 * 48 as an input neuron of the network. Kohonen neuron network is designed for two-dimensional network, with a size of 20 * 20, a total of 400 neurons.
In terms of environmental model, we randomly selected 30 images as an environment model input.
Every item's training, we only select 16 to 32 input samples. After 100 episodes of training, the training of environmental models immediately after the training, after 50 era, once an incremental study is complete, More than 40 S.
Just trained Advanced Kohonen network identification effect is not very good, but with the number of training, the Advanced Kohonen network provides a better identification result.
As shown in the figure, some demo results after learning from the Advanced Kohonen network:
For the unlike hands and fists, Advanced Kohonen networks still provide identifiable results, which is quite satisfactory. More gratifying is that this is just the result of training for a few minutes.
Fourth, conclusion
Through experiments, we believe that Advanced Kohonen networks have better adaptive and learning capabilities than feedforward networks that usually use error propagation algorithms, as a multi-class classification network, and the Advanced Kohonen network has wide application capabilities.
references:
[1] j.P.Marques de sa "Mode Identification - Principle, Method and Application"
[2] Milan Sonka, Vaclav Hlavac and Roger Boyle "Image Processing, Analysis and Machine Vision"
[3] George F. Luger "Artificial Intelligence - Structure and Strategy for Solving Complicated Problems (Fourth Edition)"
[4] Sergios THEODORIDIS AND KONSTANTINOS KOUTROUMBAS "Mode Recognition (Second Edition)"
[5] Shi Zhongzhi "Knowledge Discovery"
