Summary for lecture - Hyperparameter tuning by grid-search

Module 3. Hyperparameter tuning Automated tuning
#1

At the end of the lecture, it concludes that there is unique value optimal parameter setting can be derived from this dataset using GridSearchCV.

For now we will note that, in general, **there is no unique optimal parameter setting** : 6 models out of the 16 parameter configuration reach the maximal accuracy (up to small random fluctuations caused by the sampling of the training set).

When using .best_params_ attribute, we can get
the best set of parameters is {‘classifier__learning_rate’: 0.1, ‘classifier__max_leaf_nodes’: 30}

print(f"The best set of parameters is: "
f"{model_grid_search.best_params_}")

and looking at the heatmap constructed at the end of the lecture, when learning_rate is 0.1 and max_leaf_nodes is 30, the meat_test_score is 0.87 (it is the highest among other pair).

In this case, does the conclusion should be we are able to find the optimal best parameter using GridSearchCV method?

#2

The grid-search cross-validation took the model with the highest mean accuracy. However, the grid-search does not consider the standard deviation or the computational cost for instance.

By looking at all combinations, we observe that we could have indeed chosen any of the 6 best performing models in terms of mean accuracy, they are all equivalent.

1 Like