Since bias and variance and independent characteristics of a model, it’s possible to have both at high levels at the same time. But even understanding the intuition of the correspondence between high bias/underfitting and high variance/overfitting, I still don’t understand how a model could have both underfitting and overfitting at the same time because I associate those concepts with degree of complexity or flexibility and thus struggle to imagine examples of models (math functions) that could be simultaneously too complex/flexible and not complex/flexible enough…
Complexity and flexibility are the main factors in play, but an example of a model overfitting and underfitting at the same time can be the blood transfusion dataset, where none of the features is informative enough to help differentiate both classes. In such scenarios the model will overfit (capture noise/low test score) and underfit (low train score).
1 Like