C++ Neural Networks and Fuzzy Logic: Preface

C++ Neural Networks and Fuzzy Logic

by Valluru B. Rao

MTBooks, IDG Books Worldwide, Inc.

ISBN: 1558515526   Pub Date: 06/01/95

Previous Table of Contents Next

Adding Noise During Training

Another approach to breaking out of local minima as well as to enhance generalization ability is to introduce

some noise in the inputs during training. A random number is added to each input component of the input

vector as it is applied to the network. This is scaled by an overall noise factor, NF, which has a 0 to 1 range.

You can add as much noise to the simulation as you want, or not any at all, by choosing NF = 0. When you

are close to a solution and have reached a satisfactory minimum, you don’t want noise at that time to interfere

with convergence to the minimum. We implement a noise factor that decreases with the number of cycles, as

shown in the following excerpt from the backprop.cpp file.

// update NF

// gradually reduce noise to zero

if (total_cycles>0.7*max_cycles)

                     new_NF = 0;

else if (total_cycles>0.5*max_cycles)

                     new_NF = 0.25*NF;

else if (total_cycles>0.3*max_cycles)

                     new_NF = 0.50*NF;

else if (total_cycles>0.1*max_cycles)

                     new_NF = 0.75*NF;

backp.set_NF(new_NF);

The noise factor is reduced at regular intervals. The new noise factor is updated with the network class

function called set_NF(float). There is a member variable in the network class called NF that holds the

current value for the noise factor. The noise is added to the inputs in the input_layer member function

calc_out().

Another reason for using noise is to prevent memorization by the network. You are effectively presenting a

different input pattern with each cycle so it becomes hard for the network to memorize patterns.