Use defined scorer with other metrics

M7. Evaluating model performance M7. Exercise M7.02
#1

Hi,
Can we use the predefined scorer in this exercise with other metrics in cross_validate method ?
Thank you for your answer,

#2

It is possible to define your own scorer. You can have a look at the following documentation:

Let me know if you have a specific question regarding this documentation.

#3

When I tried combining the scorer with other metrics I got an error :
scorer =

make_scorer(score_func=precision_score,pos_label="donated")
all_scores = cross_validate(tree,data,target,cv=cv,
                           scoring=['accuracy','balanced_accuracy',scorer])
#4

Could you provide the traceback such that we can debug it.

#5

Yes sure thing
ValueError: The list/tuple elements must be unique strings of predefined scorers. One or more of the elements were callables. Use a dict of score name mapped to the scorer callable. Got ['accuracy', 'balanced_accuracy', make_scorer(precision_score, pos_label=donated)]

#6

Indeed, you cannot pass two strings and a user defined scorer.
You need to create scorers for the accuracy and balanced accuracy as well as pass 3 scorers in a list.

Otherwise no need to pass a dict like here: 3.3. Metrics and scoring: quantifying the quality of predictions — scikit-learn 1.1.0 documentation

so scoring = {"accuracy": "accuracy", "balanced_accuracy": "balanced_accuracy", "precision": make_scorer(...)}

1 Like
#7

I see, thank you for your answer