C++ Neural Networks and Fuzzy Logic: Preface

C++ Neural Networks and Fuzzy Logic

by Valluru B. Rao

MTBooks, IDG Books Worldwide, Inc.

ISBN: 1558515526   Pub Date: 06/01/95

Previous Table of Contents Next

Lyapunov Function

Neural networks are dynamic systems in the learning and training phase of their operation, and convergence is

an essential feature, so it was necessary for the researchers developing the models and their learning

algorithms to find a provable criterion for convergence in a dynamic system. The Lyapunov function,

mentioned previously, turned out to be the most convenient and appropriate function. It is also referred to as

the energy function. The function decreases as the system states change. Such a function needs to be found

and watched as the network operation continues from cycle to cycle. Usually it involves a quadratic form. The

least mean squared error is an example of such a function. Lyapunov function usage assures a system stability

that cannot occur without convergence. It is convenient to have one value, that of the Lyapunov function

specifying the system behavior. For example, in the Hopfield network, the energy function is a constant times

the sum of products of outputs of different neurons and the connection weight between them. Since pairs of

neuron outputs are multiplied in each term, the entire expression is a quadratic form.