Beginning Anomaly Detection Using

 Sparse  Autoencoders

In the above example of a simple autoencoder, the representations were only 

constrained by the size of the hidden layer (12). In such a situation, what typically 

happens is that the hidden layer is learning an approximation of PCA (principal 

component analysis). But another way to constrain the representations to be compact 

is to add a sparsity constraint on the activity of the hidden representations, so fewer 

units would fire at a given time. In Keras, this can be done by adding an activity_

regularizer to your dense layer.

The difference between the simple and sparse autoencoders is mostly due to the 

regularization term being added to the loss during training.

 

You will use the same credit card dataset as in the simple autoencoder example 

above. You will use the credit card data to detect whether a transaction is normal/

expected or abnormal/anomaly. Shown below is the data being loaded into a Pandas 

dataframe.

Figure 4-23.  Anomalies based on the threshold

Chapter 4   autoenCoders

---

141

Then, you will collect 20k normal and 400 abnormal records. You can pick different 

ratios to try, but in general more normal data examples are better because you want 

to teach your autoencoder what normal data looks like. Too much abnormal data in 

training will train the autoencoder to learn that the anomalies are actually normal, which 

goes against your goal. Split the dataframe into training and testing data sets (80- 20 split).

Now it’s time to create a neural network model with just an encoder and 

decoder layer. You will encode the 29 columns of the input credit card dataset into 

12 features using the encoder. The decoder will expand the 12 back into 29 features. 

The key difference compared to the simple autoencoder is the activity regularizer to 

accommodate the sparse autoencoder. Figure 

4-24

 shows the code to create the neural 

network.

Figure 4-24.  Code to create the neural network

Chapter 4   autoenCoders

---

142

Figure 

4-25

 shows the graph of the model as visualized by TensorBoard.