Hands-On Machine Learning with Scikit-Learn and TensorFlow

6.
Decision Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Training and Visualizing a Decision Tree 179
Making Predictions 181
Estimating Class Probabilities 183
The CART Training Algorithm 183
Table of Contents | v

---

Computational Complexity 184
Gini Impurity or Entropy? 184
Regularization Hyperparameters 185
Regression 187
Instability 189
Exercises 190
7.
Ensemble Learning and Random Forests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Voting Classifiers 193
Bagging and Pasting 196
Bagging and Pasting in Scikit-Learn 198
Out-of-Bag Evaluation 199
Random Patches and Random Subspaces 200
Random Forests 201
Extra-Trees 202
Feature Importance 202
Boosting 203
AdaBoost 204
Gradient Boosting 207
Stacking 212
Exercises 215
8.
Dimensionality Reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
The Curse of Dimensionality 218
Main Approaches for Dimensionality Reduction 219
Projection 219
Manifold Learning 222
PCA 223
Preserving the Variance 223
Principal Components 224
Projecting Down to 
d
 Dimensions 225
Using Scikit-Learn 226
Explained Variance Ratio 226
Choosing the Right Number of Dimensions 227
PCA for Compression 228
Randomized PCA 229
Incremental PCA 229
Kernel PCA 230
Selecting a Kernel and Tuning Hyperparameters 231
LLE 233
Other Dimensionality Reduction Techniques 235
Exercises 236
vi | Table of Contents

---