C++ Neural Networks and Fuzzy Logic: Preface

Adaptive Resonance Theory

ART1 is the first model for adaptive resonance theory for neural networks developed by Gail Carpenter and

Stephen Grossberg. This theory was developed to address the stability–plasticity dilemma. The network is

supposed to be plastic enough to learn an important pattern. But at the same time it should remain stable

when, in short−term memory, it encounters some distorted versions of the same pattern.

ART1 model has A and B field neurons, a gain, and a reset as shown in Figure 5.8. There are top−down and

bottom−up connections between neurons of fields A and B. The neurons in field B have lateral connections as

well as recurrent connections. That is, every neuron in this field is connected to every other neuron in this

field, including itself, in addition to the connections to the neurons in field A. The external input (or

bottom−up signal), the top−down signal, and the gain constitute three elements of a set, of which at least two

should be a +1 for the neuron in the A field to fire. This is what is termed the two−thirds rule. Initially,

therefore, the gain would be set to +1. The idea of a single winner is also employed in the B field. The gain

would not contribute in the top−down phase; actually, it will inhibit. The two−thirds rule helps move toward

stability once resonance, or equilibrium, is obtained. A vigilance parameter Á is used to determine the

parameter reset. Vigilance parameter corresponds to what degree the resonating category can be predicted.

The part of the system that contains gain is called the attentional subsystem, whereas the rest, the part that

contains reset, is termed the orienting subsystem. The top−down activity corresponds to the orienting

subsystem, and the bottom−up activity relates to the attentional subsystem.

C++ Neural Networks and Fuzzy Logic:Preface

Neocognitron

99

---

Figure 5.8

  The ART1 network.

In ART1, classification of an input pattern in relation to stored patterns is attempted, and if unsuccessful, a

new stored classification is generated. Training is unsupervised. There are two versions of training: slow and

fast. They differ in the extent to which the weights are given the time to reach their eventual values. Slow

training is governed by differential equations, and fast training by algebraic equations.

ART2 is the analog counterpart of ART1, which is for discrete cases. These are self−organizing neural

networks, as you can surmise from the fact that training is present but unsupervised. The ART3 model is for

recognizing a coded pattern through a parallel search, and is developed by Carpenter and Grossberg. It tries to

emulate the activities of chemical transmitters in the brain during what can be construed as a parallel search

for pattern recognition.

Summary

The basic concepts of neural network layers, connections, weights, inputs, and outputs have been discussed.

An example of how adding another layer of neurons in a network can solve a problem that could not be solved

without it is given in detail. A number of neural network models are introduced briefly. Learning and training,

which form the basis of neural network behavior has not been included here, but will be discussed in the

following chapter.

Previous Table of Contents Next

Copyright ©

 IDG Books Worldwide, Inc.

C++ Neural Networks and Fuzzy Logic:Preface

Summary

100

---