Beginning Anomaly Detection Using

Figure 7-10.  The new input vector weights

0.5   0.2   0.4

Filter Weights

Figure 7-11.  The new filter weights

Input:

0.5   0.2   0.4

Filter Weights:

Output:

2

8   

12

4   

6

4   2   12

1 + 2.4 + 2.4

*

=

Spacing of 1

5.8

Figure 7-12.  Calculating the first entry in the output factor using dilated 

one- dimensional convolutions with a dilation factor of two

Chapter 7   temporal Convolutional networks

---

265

Input:

0.5   0.2   0.4

Filter Weights:

Output:

2   

8

12

4

6   

4

2   12

4 + 0.8 + 1.6

*

=

5.8

6.4

Figure 7-13.  The next set of three input vector values are multiplied with the filter 

weights to produce the next output vector value

Input:

0.5   0.2   0.4

Filter Weights:

Output:

2   8   

12

4   

6

4   

2

12

6 + 1.2 + 0.8

*

=

5.8   6.4

8

Figure 7-14.  The third set of three input vector values are multiplied with the filter 

weights to produce the next output vector value

Input:

0.5   0.2   0.4

Filter Weights:

Output:

2   8   12

4

6   

4

2   

12

2 + 0.8 + 4.8

*

=

5.8   6.4   8

7.6  

Figure 7-15.  The final set of three input vector values are multiplied with the filter 

weights to produce the last output vector value

Chapter 7   temporal Convolutional networks

---

266

Now that we’ve covered what a 

dilated convolution looks like in the context of one- 

dimensional convolutions, let’s look at the difference between an 

acausal and a casual 

dilated convolution. To illustrate this concept, assume that both examples are referring 

to a set of dilated one-dimensional convolutional layers. With that in mind, Figure 

7-16

 

shows an 

acausal network.

It might not be that apparent from the way the architecture is structured, but if you 

think of the input layer as a sequence of some data going forward in time, you might be 

able to see that information from the future would be accounted for when selecting the 

output. In a 

casual network, we only want information that we’ve learned up until the 

present, so none of the information from the future will be accounted for in the model’s 

predictions. Figure 

7-17

 shows what a 

causal network looks like.

Input Layer

Hidden Layer 1

Hidden Layer 2

Output Layer

Figure 7-16.  An acausal dilated network. The first hidden layer has a dilation 

factor of two, and the second hidden layer has a dilation factor of four. Notice how 

inputs “forward in the sequence” contribute to the next layer’s node as well

Chapter 7   temporal Convolutional networks

---

267

From this, we can see how the linear nature of time is preserved in the model, and 

how no information from the future would be learned by the model. In casual networks, 

only information from the past until the present is considered by the model. The dilated 

temporal convolutional network we are referring to has a similar model architecture, 

utilizing dilated causal convolutions in each layer preceding the output layer.