Beginning Anomaly Detection Using

Figure 3-4.  This is how dot product works. Shown here is an example with two 

different types of vector notation

Figure 3-5.  An equation that captures the basic functionality of an artificial 

neuron. In this case, f(x) is an activation function

Chapter 3   IntroduCtIon to deep LearnIng

---

77

A 

hidden layer is one that is between the input layer and the output layer. There 

can be multiple hidden layers in a network. Now that you’ve seen what an artificial 

neural network can look like, let’s take a look at how the data can flow through this 

network. First, we start with nothing but the input data in the network, and assume that 

neurons only wholly activate (neurons can partially activate depending on the activation 

function, but in this example each neuron either outputs a 1 or a 0) (Figure 

3-7

).

Figure 3-6.  An example of what an artificial neural network can look like

Chapter 3   IntroduCtIon to deep LearnIng

---

78

The input layer takes all of the corresponding inputs and produces an output that 

is linked to the first hidden layer. The outputs of the nodes that activate in the input 

layer are now the inputs of the hidden layer, and the new data flows correspondingly 

(Figure 

3-8

).

Figure 3-7.  The input data runs through the input layer, and selective nodes fire 

based on the input received

Chapter 3   IntroduCtIon to deep LearnIng

---

79

Hidden layer 1 processes the data in a similar fashion to the input layer, just with 

different parameters for activation function, weight, bias, etc. The data passes through 

this layer and the output of this layer becomes the input for the next hidden layer. In this 

case, only two nodes activate based on the input from the previous layer (Figure 

3-9

).

Figure 3-8.  The outputs from the activated neurons in the input layer pass on 

to the first hidden layer. These outputs are now the inputs of the next layer, and 

selective neurons fire based on this input

Chapter 3   IntroduCtIon to deep LearnIng

---

80

Hidden layer 2 processes the data and sends the data to a new layer called the output 

layer, where only one of the nodes in the layer will be activated. In this case, the first 

node in the output layer is activated (Figure 

3-10

).

Figure 3-9.  This process repeats with hidden layer 2

Chapter 3   IntroduCtIon to deep LearnIng

---

81

The nodes in the output layer can represent the different labels that you want to give 

to the input data. For example, in the iris dataset, you can take various measurements of 

an iris flower and train an artificial neural network on this data to classify the species of 

the flower.

Upon initialization, the weights of the model will be far from ideal. Throughout the 

training process, the data flow from the model goes forward (left to right from input to 

output), and then backwards in what is known as