Python Artificial Intelligence Projects for Beginners

[ 83 ]
Training proceeds in batches, which means that several training cases are sent through the
network and the outputs, called 
predictions
, are collected. Then, the loss is computed for
each batch, which is the measure of the overall error:
6TCKPKPIRTQEGFWTGGXCNWCVGCDCVEJCFLWUVYGKIJVUCPFTGRGCV
Each weight in the network is then adjusted depending on whether and how much that
weight contributed to the overall loss. With very gradual adjustments, it should be the case
that when examples in this batch are visited again, predictions will be more accurate.
The network is often trained over several epochs. By an epoch, we mean all the training
data having been processed once. So, 10 epochs means looking at the same training data 10
times. We often segregate 20% or so of the training data as a validation set. This is data that
we don't use during training and instead only use to evaluate the model after each epoch.
Ideally, we want the network to become more accurate, which means we want to decrease
loss, and this should be true for both the training set and the validation set.

---

Neural Networks
Chapter 4
[ 84 ]
The following set of graph shows this ideal kind of behavior:
+FGCNDGJCXKQT
Note the signs of overfitting, meaning training loss goes down but validation loss goes up.
If the network is not designed correctly, for example, if it has too many layers, the network
may overfit, meaning it performs very well in the training set but poorly on the validation
set. This is an issue because ultimately we want to use the neural network on new data
from the real world, which will probably be a little different than the training set, hence we
use a validation set to see how well the network performs on data it didn't see for training.