In module 2., you introduce and often refer to the relation between score and error. What I got is that score is (computed by taking) the opposite of error.
Why is it so? Why isn’t the error computed as 1 - score instead ? Indeed, being between 0 and 1, score seems to me like a probability measure.
The reason is due to some historical choices in the design of the GridSearchCV and RandomizedSearchCV by scikit-learn developers.
These SearchCV will select the set of hyperparameters that maximizes a given score. An error is something that is in the range [0, inf). Therefore, using it instead of a score in a SearchCV, one would need to minimize it.
Instead of switching minimization and maximization depending if we use an error or a score, the SearchCV always request a score (a higher value means a better model). Therefore, a choice was made to take the negative error that would range from (-inf, 0] that can therefore be maximized.
Thanks for the detailed info @glemaitre58
However, my question was about something simpler (I guess). That’s on me, I was not clear enough most probably.
Often, in module 2. (at the beginning, but also when talking about the “validation curve”) the code does something like cv_results["test_error"] = - cv_results["test_score"] from which one can deduce that it is always the case that (test_)error = - (test_)score. My question is then, why isn’t it computed like (test_)error = 1 - (test_)score (since (test_)score \in [0, 1], I would say)?
It boils down to my previous explanation. For instance, when we use neg_mean_absolute_error for scoring, scikit-learn is defining this score as the negative mean absolute error. In short, scikit-learn does something like:
from sklearn.metrics import mean_absolute_error
neg_mean_absolute_error = -1 * mean_absolute_error(y_true, y_pred)
Therefore, we just revert to get the mean absolute error. 1 - score would not provide the expected mean absolute error.