Beginning Anomaly Detection Using

lr: A float value specifying the learning rate. Default = 0.01 (or 1e-2).

• 

momentum: (Optional). Some float value specifying the momentum 

factor. This parameter helps accelerate the optimization steps in the 

direction of the optimization, and helps reduce oscillations when 

the local minimum is overshot (refer to Chapter 

3

 to refresh your 

understanding on how a loss function is optimized). Default = 0.

• 

alpha: (Optional) A smoothing constant. Default = 0.99

• 

eps: (Optional). Some float value where epsilon e >= 0. Epsilon is 

some small number, described as 10E-8 in the paper, to help prevent 

division by 0. Default is 1e-8.

• 

centered: (Optional) If True, then compute the centered RMSprop 

and have the gradient normalized by an estimation of its variance. 

Default = False.

• 

weight_decay: An l2_penalty for weights that are too high, helping 

incentivize smaller model weights. Default = 0.

Hopefully by now you understand how PyTorch works by looking at some of the 

functionality that it offers. You built and applied a model to the MNIST data set in an 

organized format, and you looked at some of the basics of PyTorch by learning about the 

layers, how models are constructed, how activations are performed, and what the loss 

functions and optimizers are.