Beginning Anomaly Detection Using

 Deep  Autoencoders

You do not have to limit yourself to a single layer as encoder or decoder; you can use a 

stack of layers. It’s not a good idea to use too many hidden layers, and how many layers 

depends on the use case, so you have to play with it to seek the optimal number of layers 

and the compressions.

The only thing that really changes is the number of layers. Shown below is the simple 

autoencoder with multiple layers.

You will use the example of credit card data to detect whether a transaction is 

normal/expected or abnormal/anomaly. Shown below is the data being loaded into 

Pandas dataframe.

You will collect 20k normal and 400 abnormal records. You can pick different ratios 

to try, but in general more normal data examples are better because you want to teach 

your autoencoder what normal data looks like. Too much abnormal data in training 

will train the autoencoder to learn that the anomalies are actually normal, which goes 

against your goal. Split the dataframe into training and testing data sets (80-20 split).

Now it’s time to create a deep neural network model with three layers for the encoder 

layer and three layers as part of decoder layer. You will encode the 29 columns of the 

input credit card dataset into 16, then 8, and then 4 features using the encoder. The 

decoder expands the 4 back into the 8 and then 16 and then finally into 29 features. 

Figure 

4-26

 shows the code to create the neural network.

Figure 4-25.  Model graph created by TensorBoard

Chapter 4   autoenCoders

---

143

Figure 

4-27

 shows the graph of the model as visualized by TensorBoard.