Beginning Anomaly Detection Using

 What Is an LSTM?

A LSTM network is a kind of recurrent neural network. As seen above, a recurrent 

neural network is a neural network that attempts to model time or sequence dependent 

behavior, such as language, stock prices, weather sensors, and so on. This is performed 

by feeding back the output of a neural network layer at time T to the input of the 

same network layer at time T + 1. LSTM builds on top of the RNN, adding a memory 

component meant to help propagate the information learned at a time T to the future 

T+1, T+2, and so on. The main idea is that LSTM can forget irrelevant parts of previous 

state while selectively updating state and then outputting certain parts of the state that 

are relevant to the future.

How does this solve the vanishing gradient problem in RNNs? Well, now we are 

throwing some state, updating some state, and propagating forward some part of the 

state so we no longer have a long chain of backpropagation seen in RNNs. Thus, LSTMs 

are much more efficient than typical RNN.

Chapter 6   Long Short-term memory modeLS 

---

219

Figure 

6-7

 is a RNN with tanh activation.

The tanh function is called an activation function. There are several types of 

activation functions that help in applying non-linear transformations on the inputs at 

every node in the neural network. Figure 

6-8

 shows common activation functions.