Hi,
What do you mean by “make them vote” in the sentence “From a deployment point of view, one could also choose to deploy all the models found by the outer cross-validation loop and make them vote to get the final predictions.” Vote = take the mean?
The concept of voting is explained more in detail in Module 6. For the moment imagine that you find 3 different combination of learning_rate and max_leaf_nodes that lead to more or less the same generalization performance. Then this paragraph says that you could train 3 different models (one per each combination) and ask them to predict the class of an unseen sample. If a majority of such models predict the same given class, then that is the class predicted by the “ensemble of models”.
Ok. And what about for regressor?
Ah! Then yes, the average of their votes.
Taking the average prediction of regressors is indeed the most common to combine predictions.
For classifiers, it can either be an hard-vote on predicted discrete class labels or averaging the output values of the predict_proba method when available.