Beginning Anomaly Detection Using

Boolean/

Bernoulli RBM. Figure 

5-2

 shows an RBM.

v

0

v

1

v

2

h

4

h

3

h

2

h

0

h

1

Input 

Layer

Hidden 

Layer

Figure 5-2.  A visual representation of a basic restricted Boltzmann machine

We can expand this model out even more to include biases (see Figure 

5-3

).

Chapter 5   Boltzmann maChines

---

182

Bias 

a adds to all of the outputs of the input layer, and bias b adds to the outputs of 

the hidden layer. From here, we can define what is called the 

energy function, which the 

RBM tries to minimize. The 

energy function is shown in Figure 

5-4

.

v

0

v

1

v

2

h

4

h

3

h

2

h

0

h

1

Input 

Layer

Hidden 

Layer

b

a

Bias b

Bias a

Figure 5-3.  A visual representation of a restricted Boltzmann machine with a 

different bias feeding into each of the two layers

Figure 5-4.  A formula that defines the energy function of the restricted Boltzmann 

machine

The first summation term is an element-wise multiplication between bias 

a and 

visible layer 

v, where each term a

i

 is multiplied with each term v

i

. The second summation 

term follows the same logic, except uses element-wise multiplication with bias 

b and 

hidden layer 

h. Finally, the last summation term multiplies each visible node v

i

 with 

each hidden node h

j

 and the weight value w

ij

 for that connection.

The summations are basically element-wise multiplication between two vectors, one 

being 

transposed, so 1xn (1 column n rows), and the other being nx1 (n columns 1 

row). When a vector or matrix is transposed, we reverse the dimensions of the vector/

matrix and rearrange the values. In a vector, the same values in a row/column are now in 

a column/row. For matrices, it’s a bit more complex. To better understand the concept of 

transposing a vector or matrix, refer to Figure 

5-5

, Figure 

5-6

, and Figure 

5-7

.

Chapter 5   Boltzmann maChines

---

183