C++ Neural Networks and Fuzzy Logic: Preface

The Sigmoid Function

More than one function goes by the name sigmoid function. They differ in their formulas and in their ranges.

They all have a graph similar to a stretched letter s. We give below two such functions. The first is the

hyperbolic tangent function with values in (–1, 1). The second is the logistic function and has values between

0 and 1. You therefore choose the one that fits the range you want. The graph of the sigmoid logistic function

is given in Fig. 5.3.

1.  f(x) = tanh(x) = ( e

x

 − e

−x

) / (e

x

 + e

−x

)

2.  f(x) = 1 / (1+ e

−x

)

Note that the first function here, the hyperbolic tangent function, can also be written, as 1 − 2e

−x

 / (e

x

 + e

−x

 )

after adding and also subtracting e

−x

 to the numerator, and then simplifying. If now you multiply in the second

term both numerator and denominator by e

x

, you get 1 − 2/ (e

2x

 + 1). As x approaches −, this function goes to

−1, and as x approaches +, it goes to +1. On the other hand, the second function here, the sigmoid logistic

function, goes to 0 as x approaches −, and to +1 as x approaches +. You can see this if you rewrite 1 / (1+

e

−x

) as 1 − 1 / (1+ e

x

), after manipulations similar to those above.

C++ Neural Networks and Fuzzy Logic:Preface

Outputs

94

---

You can think of equation 1 as the bipolar equivalent of binary equation 2. Both functions have the same

shape.

Figure 5.3 is the graph of the sigmoid logistic function (number 2 of the preceding list).