[QUESTION] SVC's default value for `decision_function_shape` about handson-ml3 HOT 2 CLOSED

Hi @liganega ,

Thanks for your feedback. Are you referring to the 2nd edition? Because there's already the following explanation in the 3rd edition:

This code actually made 45 predictions—one per pair of classes—and it selected the class that won the most duels. If you call the decision_function() method, you will see that it returns 10 scores per instance: one per class. Each class gets a score equal to the number of won duels plus or minus a small tweak (max ±0.33) to break ties, based on the classifier scores.

Indeed, under the hood, SVC always uses OvO, there's no way to change that. The decision_function_shape hyperparameter only affects the output of the decision_function() method: if this hyperparameter is set to "ovr" (which is the default), then it's still going to use OvO under the hood, but after that it will aggregate the 45 scores into 10 (one per class), while if you set this hyperparameter to "ovo", it will just return the 45 scores directly.

I think this is a confusing part of Scikit-Learn's API. Many people were confused when first reading the docs (myself included): it really looks like the class uses OvR by default, but it doesn't. It still trains 45 models, and it still makes 45 predictions, it's just the way these results are presented. More details here.

Hope this helps.

from handson-ml3.

The confusion comes from the number "10".

The number "10" in the sentence below does NOT correspond to the "10" binary classification models which would be trained if "OvR" method should really be applied. It's just the number of classes. And the 10 scores are aggregates of the 45 scores leared from 45 binary classification models based on the "OvO" method.

it returns 10 scores per instance: one per class

On the other hand, when the following hyperparameter is set, then 45 scores will be returned.

svm_clf.decision_function_shape = "ovo"

So IMHO, the confusion can easily arise.

Anyway, it is a subtle point, but well understood now. Thank you.

from handson-ml3.