Beginning Anomaly Detection Using

decay: Some float value where the decay d >= 0. Helps determine 

how much the learning rate decays by after each update (so that as 

the local minimum is approached, or after some number of training 

iterations, the learning rate decreases so smaller step sizes are taken. 

Big learning rates means the local minimum might be overshot more 

easily).

• 

amsgrad: A Boolean on whether or not to apply the AMSGrad 

version of this algorithm. For more details on the implementation 

of this algorithm, check out “On the Convergence of Adam and 

Beyond.”

 RMSprop

keras.optimizers.RMSprop()

RMSprop is a good algorithm for recurrent neural networks. RMSprop is a gradient- 

based optimization technique developed to help address the problem of gradients 

becoming too large or too small. RMSprop helps combat this problem by normalizing 

the gradient itself using the average of the squared gradients. In Chapter 

7

, it’s explained 

that one of the problems with RNNs is the vanishing/exploding gradient problem, 

leading to the development of LSTMs and GRU networks. And so it’s of no surprise that 

RMSprop pairs well with recurrent neural networks.

Appendix A   intro to KerAs

---

348

Besides the learning rate, it’s recommended to leave the rest of the algorithms in 

their default settings. With that in mind, here are the parameters for this optimizer:

•