Beginning Anomaly Detection Using

decay: Some float value where the decay d >= 0. Helps determine 

how much the learning rate decays by after each update (so that as 

the local minimum is approached, or after some number of training 

iterations, the learning rate decreases so smaller step sizes are taken. 

Big learning rates means the local minimum might be overshot more 

easily).

• 

nesterov: A Boolean value to determine whether or not to apply 

Nesterov momentum. Nesterov momentum is a variation of 

momentum where the gradient is computed not from the current 

position, but from a position that takes into account the momentum. 

This is because the gradient always points in the right direction, 

but the momentum might carry the position too far forward and 

overshoot. Since it doesn’t use the current position but instead 

some intermediate position that takes into account momentum, the 

gradient from that position can help correct the current course so 

that the momentum doesn’t carry the new weights too far forward.

It essentially helps for more accurate weight updates and helps converge faster.

 Adam

keras.optimizers.Adam()

The Adam optimizer is an algorithm that extends upon SGD, and has grown quite 

popular in deep learning applications in computer vision and in natural language 

processing.

Appendix A   intro to KerAs

---

347

These are the parameters for the algorithm:

•