Hands-On Machine Learning with Scikit-Learn and TensorFlow

3.
Classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
MNIST 89
Training a Binary Classifier 92
Performance Measures 92
Measuring Accuracy Using Cross-Validation 93
Confusion Matrix 94
Precision and Recall 96
Precision/Recall Tradeoff 97
The ROC Curve 101
Multiclass Classification 104
Error Analysis 106
Multilabel Classification 110
Multioutput Classification 111
Exercises 112
4.
Training Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Linear Regression 116
The Normal Equation 118
iv | Table of Contents

---

Computational Complexity 121
Gradient Descent 121
Batch Gradient Descent 125
Stochastic Gradient Descent 128
Mini-batch Gradient Descent 130
Polynomial Regression 132
Learning Curves 134
Regularized Linear Models 138
Ridge Regression 138
Lasso Regression 141
Elastic Net 143
Early Stopping 144
Logistic Regression 145
Estimating Probabilities 145
Training and Cost Function 147
Decision Boundaries 148
Softmax Regression 151
Exercises 155
5.
Support Vector Machines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Linear SVM Classification 157
Soft Margin Classification 158
Nonlinear SVM Classification 161
Polynomial Kernel 162
Adding Similarity Features 163
Gaussian RBF Kernel 164
Computational Complexity 165
SVM Regression 166
Under the Hood 168
Decision Function and Predictions 168
Training Objective 169
Quadratic Programming 171
The Dual Problem 172
Kernelized SVM 173
Online SVMs 176
Exercises 177