Question 1 : LeaveOneGroupOut

I don’t see why Leave One Group Out could’nt be a good way to evaluate the ability of the model to make good predictions on patients from unseen hospitals?

If classes are not balanced each score could be very different but the test of unseen unbalanced hospital would me more realistic than in a GroupKFold ?

LeaveOneGroupOut is indeed an adequate option to use. But be aware that the question asks for the conventional LeaveOneOut cross-validation (which would not take into account the group structure).

Indeed, I haven’t seen it was LeaveOneOut and not LeaveOneGroupOut…
Thank you