Beginning Anomaly Detection Using

Figure B-21.  The ReLU layer in the forward function of a model

 Softmax

torch.nn.Softmax()

This performs a softmax on the given dimension.

The general formula for softmax is shown in Figure 

B-22

 (K is the number of 

samples).

Here is the parameter:

• 

dim: The dimension to compute softmax along, determined by some 

integer 

n. This is so every slice along the dimension will sum to 1. 

Default = None.

You can define this as a layer within the model itself, or apply softmax in the forward 

function like so:

torch.nn.functional.softmax(input, dim=None, _stacklevel=3)

appendix B   intro to pytorch

---

385

Input is the previous layer.

Figure 

B-23

 shows an example of how you can use this layer in the forward function.