Beginning Anomaly Detection Using

 Anomaly Detection with OC-SVM

Now that you know more about how SVMs work, let’s get started by applying a one-class 

SVM to the KDDCUP 1999 data set.

Import your modules and load up the data set (see Figure 

2-41

 and Figure 

2-42

).

LPSRUW

QXPS\

DV

QS

LPSRUW

SDQGDV

DV

SG

LPSRUW

PDWSORWOLES\SORW

DV

SOW

IURP

VNOHDUQPRGHOBVHOHFWLRQ

LPSRUW

WUDLQBWHVWBVSOLW

IURP

VNOHDUQSUHSURFHVVLQJ

LPSRUW

/DEHO(QFRGHU

IURP

VNOHDUQVYP

LPSRUW

2QH&ODVV690



PDWSORWOLELQOLQH

Figure 2-41.  Importing your modules for the OC-SVM

FROXPQV >GXUDWLRQSURWRFROBW\SHVHUYLFHIODJVUFBE\WHV

GVWBE\WHVODQGZURQJBIUDJPHQWXUJHQW

KRWQXPBIDLOHGBORJLQVORJJHGBLQQXPBFRPSURPLVHG

URRWBVKHOOVXBDWWHPSWHGQXPBURRW

QXPBILOHBFUHDWLRQVQXPBVKHOOVQXPBDFFHVVBILOHV

QXPBRXWERXQGBFPGVLVBKRVWBORJLQ

LVBJXHVWBORJLQFRXQWVUYBFRXQWVHUURUBUDWH

VUYBVHUURUBUDWHUHUURUBUDWHVUYBUHUURUBUDWH

VDPHBVUYBUDWHGLIIBVUYBUDWHVUYBGLIIBKRVWBUDWH

GVWBKRVWBFRXQWGVWBKRVWBVUYBFRXQW

GVWBKRVWBVDPHBVUYBUDWHGVWBKRVWBGLIIBVUYBUDWH

GVWBKRVWBVDPHBVUFBSRUWBUDWHGVWBKRVWBVUYBGLIIBKRVWBUDWH

GVWBKRVWBVHUURUBUDWHGVWBKRVWBVUYBVHUURUBUDWH

GVWBKRVWBUHUURUBUDWHGVWBKRVWBVUYBUHUURUBUDWHODEHO@

GI SGUHDGBFVY GDWDVHWVNGGBFXSBNGGFXSGDWDNGGFXSGDWDFRUUHFWHG

VHS QDPHV FROXPQVLQGH[BFRO 1RQH

Figure 2-42.  Defining the columns for the data set, and importing the data set 

into the data frame variable df

Chapter 2   traditional Methods of anoMaly deteCtion

---

64

Now, let’s move on to filtering out all the normal data entries. You will make two 

data frames that consist of normal entries and an equal mix of anomalies and normal 

data entries.

Run the code in Figure 

2-43

.

Figure 

2-44

 shows the shapes of the two data frames.

The first half of the data frame “novelties” consists of anomalies, while the latter half 

consists of normal data entries.

Now you move on to encoding all the categorical values in the data frames (see 

Figure 

2-45

).

GI GI>GI>VHUYLFH@  KWWS@

GI GIGURS VHUYLFHD[LV 

FROXPQVUHPRYH VHUYLFH

QRYHOWLHV GI>GI>ODEHO@ QRUPDO@

QRYHOWLHVBQRUPDO GI>@

QRYHOWLHV SGFRQFDW >QRYHOWLHVQRYHOWLHVBQRUPDO@

QRUPDO GI>GI>ODEHO@  QRUPDO@

Figure 2-43.  Filtering out the anomalies and the normal data points to construct 

a new data set that is a mixture of the two

Figure 2-44.  Printing out the shapes of the novelty and normal data sets

Chapter 2   traditional Methods of anoMaly deteCtion

---

65

Now run the code in Figure 

2-46

 to set up your training, testing, and validation sets.

Figure 

2-47

 shows the shapes of the data sets.