Hands-On Machine Learning with Scikit-Learn and TensorFlow

102 | Chapter 3: Classification

---

Let’s train a 
RandomForestClassifier
and compare its ROC curve and ROC AUC
score to the 
SGDClassifier
. First, you need to get scores for each instance in the
training set. But due to the way it works (see 
Chapter 7
), the 
RandomForestClassi
fier
class does not have a 
decision_function()
method. Instead it has a 
pre
dict_proba()
method. Scikit-Learn classifiers generally have one or the other. The
predict_proba()
method returns an array containing a row per instance and a col‐
umn per class, each containing the probability that the given instance belongs to the
given class (e.g., 70% chance that the image represents a 5):
from
sklearn.ensemble
import
RandomForestClassifier
forest_clf
=
RandomForestClassifier
(
random_state
=
42
)
y_probas_forest
=
cross_val_predict
(
forest_clf
, 
X_train
, 
y_train_5
, 
cv
=
3
,
method
=
"predict_proba"
)
But to plot a ROC curve, you need scores, not probabilities. A simple solution is to
use the positive class’s probability as the score:
y_scores_forest
=
y_probas_forest
[:, 
1
]
# score = proba of positive class
fpr_forest
, 
tpr_forest
, 
thresholds_forest
=
roc_curve
(
y_train_5
,
y_scores_forest
)
Now you are ready to plot the ROC curve. It is useful to plot the first ROC curve as
well to see how they compare (
Figure 3-7
):
plt
.
plot
(
fpr
, 
tpr
, 
"b:"
, 
label
=
"SGD"
)
plot_roc_curve
(
fpr_forest
, 
tpr_forest
, 
"Random Forest"
)
plt
.
legend
(
loc
=
"lower right"
)
plt
.
show
()
Figure 3-7. Comparing ROC curves