C++ Neural Networks and Fuzzy Logic: Preface

Another Example of Using Backpropagation

Before modifying the simulator to add features, let’s look at the same problem we used the Kohonen map to

analyze in Chapter 12. As you recall, we would like to be able to distinguish alphabetic characters by

assigning them to different bins. For backpropagation, we would apply the inputs and train the network with

anticipated responses. Here is the input file that we used for distinguishing five different characters, A, X, H,

B, and I:

0 0 1 0 0  0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1

1 0 0 0 1  0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0

1 0 0 0 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

1 1 1 1 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

1 0 0 0 1

1 0 0 0 1

1 0 0 0 1

1 1 1 1 1

0 0 1 0 0

Each line has a 5x7 dot representation of each character. Now we need to name each of the output categories.

We can assign a simple 3−bit representation as follows:

A

000

C++ Neural Networks and Fuzzy Logic:Preface

Chapter 13 Backpropagation II

258

---

X

010

H

100

B

101

I

111

Let’s train the network to recognize these characters. The training.dat file looks like the following.

0 0 1 0 0  0 1 0 1 0  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1

1 0 0 0 1  0 1 0 1 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 1 0 1 0

1 0 0 0 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

1 1 1 1 1  1 0 0 0 1  1 0 0 0 1  1 1 1 1 1  1 0 0 0 1  1 0 0 0 1

0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0  0 0 1 0 0

1 0 0 0 1  0 0 0

1 0 0 0 1  0 1 0

1 0 0 0 1  1 0 0

1 1 1 1 1  1 0 1

0 0 1 0 0  1 1 1

Now you can start the simulator. Using the parameters (beta = 0.1, tolerance = 0.001, and max_cycles = 1000)

and with three layers of size 35 (input), 5 (middle), and 3 (output), you will get a typical result like the

following.

−−−−−−−−−−−−−−−−−−−−−−−−−− −

         done:   results in file output.dat

                        training: last vector only

                        not training: full cycle

                        weights saved in file weights.dat

−−>average error per cycle = 0.035713<−−

−−>error last cycle = 0.008223 <−−

−>error last cycle per pattern= 0.00164455 <−−

−−−−−−>total cycles = 1000 <−−

−−−−−−>total patterns = 5000 <−−

−−−−−−−−−−−−−−−−−−−−−−−−−−−

The simulator stopped at the 1000 maximum cycles specified in this case. Your results will be different since

the weights start at a random point. Note that the tolerance specified was nearly met. Let us see how close the

output came to what we wanted. Look at the output.dat file. You can see the match for the last pattern as

follows:

for input vector:

0.000000  0.000000  1.000000  0.000000  0.000000  0.000000  0.000000

1.000000  0.000000  0.000000  0.000000  0.000000  1.000000  0.000000

0.000000  0.000000  0.000000  1.000000  0.000000  0.000000  0.000000

0.000000  1.000000  0.000000  0.000000  0.000000  0.000000  1.000000

0.000000  0.000000  0.000000  0.000000  1.000000  0.000000  0.000000

output vector is:

0.999637  0.998721  0.999330

expected output vector is:

1.000000  1.000000  1.000000

−−−−−−−−−−−

C++ Neural Networks and Fuzzy Logic:Preface

Chapter 13 Backpropagation II

259

---

Previous Table of Contents Next

Copyright ©

 IDG Books Worldwide, Inc.

C++ Neural Networks and Fuzzy Logic:Preface

Chapter 13 Backpropagation II

260

---