Hello,
as we have seen in a previous lecture (see M2, and reply by Mr Lemaitre), dependent on how many data points we have and on the noise in the data, there may be cases where the fit is bad even if the model is the true model and vice versa. So by chance, there may be a case when we have data points that may be separable linearly although it is not the true data-generating process, isn’t it?
Thank you!
If it is not the data-generating process then a linear model will not be able to generalize to new unseen data.
Thank you. However, the question refers to the training set?
This may only happen if you have very few data points in the training set such that the true data-generating process is indeed not represented by those points. Assuming this is the case (though trying to infer a model from few data points would in general be a bad practice), looking for the cross-validated training error would either lead to non-perfect scores or models that would differ considerably between each fold to be taken seriously.