Understanding of this para

M6. Ensemble of models M6. Hyperparameter tuning
#1

The mean test score in the held-out test set is slightly better than the score of the best model. The reason is that the final model is refitted on the whole training set and therefore, on more data than the inner cross-validated models of the grid search procedure.

Held out test set=Validation set?,
Best model=? Random Forest or something else?,
Boosting takes whole training set why is it mentioned specifically about whole training set?,
Inner cross validation is not done here what does it mean then?

Please explain @glemaitre58

#2

Yes

The model provided in the grid-search with the combination of parameter providing the best test mean score

When passing X_train and y_train, these datasets will be split into new training and validation set internally to be able to fit and score with the model (using a CV). The “whole training set” means that we fit again the model with the best combination of parameters on X_train/y_train and not the inner training sets.

The inner cross-validation is done internally of the grid-search. That’s why the model is called GridSearchCV and has a cv parameter. So cross_validate makes the outer cross-validation and the GridSearchCV makes the inner cross-validation.

#3

Actually, no. This is something similar as shown in the figure "KFold cross-validation inside (non-shuffled-)train-test split in the Evaluation and hyperparameter tuning notebook. By held-out test set we mean the test set (blue samples from said figure), not the validation set (green samples in such figure).

#4

Ups, I did not see the world test. Indeed, @ArturoAmorQ is right.

#5

Thanks @glemaitre58 @ArturoAmorQ