C++ Neural Networks and Fuzzy Logic: Preface

The Threshold Function

The output of any neuron is the result of thresholding, if any, of its internal activation, which, in turn, is the

weighted sum of the neuron’s inputs. Thresholding sometimes is done for the sake of scaling down the

activation and mapping it into a meaningful output for the problem, and sometimes for adding a bias.

Thresholding (scaling) is important for multilayer networks to preserve a meaningful range across each layer’s

operations. The most often used threshold function is the sigmoid function. A step function or a ramp

function or just a linear function can be used, as when you simply add the bias to the activation. The sigmoid

function accomplishes mapping the activation into the interval [0, 1]. The equations are given as follows for

the different threshold functions just mentioned.