C++ Neural Networks and Fuzzy Logic: Preface

Training and Convergence

Suppose you have a criterion such as energy to be minimized or cost to be decreased, and you know the

optimum level for this criterion. If the network achieves the optimum value in a finite number of steps, then

you have convergence for the operation of the network. Or, if you are making pairwise associations of

patterns, there is the prospect of convergence if after each cycle of the network operation, the number of errors

is decreasing.

It is also possible that convergence is slow, so much so that it may seem to take forever to achieve the

convergence state. In that case, you should specify a tolerance value and require that the criterion be achieved

within that tolerance, avoiding a lot of computing time. You may also introduce a momentum parameter to

further change the weight and thereby speed up the convergence. One technique used is to add a portion of the

previous change in weight.

Instead of converging, the operation may result in oscillations. The weight structure may keep changing back

and forth; learning will never cease. Learning algorithms need to be analyzed in terms of convergence as

being an essential algorithm property.

Previous Table of Contents Next

Copyright ©

 IDG Books Worldwide, Inc.

C++ Neural Networks and Fuzzy Logic:Preface

Training and Convergence

109

---