Beginning Anomaly Detection Using

 Cross  Entropy

torch.nn.CrossEntropyLoss()

The equation is shown in Figure 

B-29

.

Figure B-29.  The general formula for cross entropy loss

In this case, 

n is the number of samples in the whole data set. The parameter h

θ

 

represents the model with the weight parameter 

θ passed in, so h

θ

(x

i

) would give the 

predicted value for x

i

 with model’s weights 

θ. Finally, y

i

 represents the true labels for 

data point at index i. The data needs to be regularized to be between 0 and 1, so for 

categorical cross entropy, it must be piped through a softmax activation layer.  

The categorical cross entropy loss is also called 

softmax loss.

Equivalently, you can write the previous equation as Figure 

B-30

.

appendix B   intro to pytorch

---

389

In this case, 

m is the number of classes.

The categorical cross entropy loss is a commonly used metric in classification tasks, 

especially in computer vision with convolutional neural networks.

This function has several parameters (two are deprecated):

•