Beginning Anomaly Detection Using

for

f

in

range(0, 20):

normal = normal.iloc[np.random.permutation(

len

(normal))]

data_set = pd.concat([normal[:2000], anomalies])

x_train, x_test = train_test_split(data_set, test_size = 0.4, 

random_state = 42)

x_train = x_train.sort_values(by=['Time'])

x_test = x_test.sort_values(by=['Time'])

y_train = x_train["Class"]

y_test = x_test["Class"]

x_train.head(10)

Figure 7-28.  Defining the training and testing sets and sorting both by time to 

maintain the temporal flow

Chapter 7   temporal Convolutional networks

---

274

Shuffling the normal data set as well as using the train_test_split function to 

randomly select testing and training samples helps ensure that you pick a good range of 

data values to represent normal data. You can limit the number of iterations in the for 

block at the start of the code if you wish.

From there, the first 10,000 data entries of the shuffled normal data are concatenated 

with the anomalies, and the training and testing data sets are created. Both sets are then 

sorted by the Time column to maintain the entire aspect of time.

The output should look somewhat like the Figure 

7-29

.

Notice how the indices vary in number, although they are all ordered by time.

Now you can move on to reshaping your data sets to pass into the model.

Running the code block in Figure 

7-30

 can give you a sense of how the data sets are 

structured.

Figure 7-29.  The data sets sorted by the Time column

Chapter 7   temporal Convolutional networks

---

275

The output should look somewhat like Figure 

7-31

.

To pass the data sets into the model, the x sets must be three-dimensional, and the y 

sets must be two-dimensional. You can simply reshape the x sets, and change the y sets 

to be categorical (refer to Chapter 

3

 to see what the keras to_categorical() function does).

Run the code in Figure 

7-32

.

SULQW

6KDSHV?Q[BWUDLQV?Q\BWUDLQV?Q [BWUDLQVKDSH

\BWUDLQVKDSH

SULQW

[BWHVWV?Q\BWHVWV?Q [BWHVWVKDSH

\BWHVWVKDSH