Is it possible to get a perfect fit (zero prediction error on the training set) with a linear classifier on a non-linearly separable dataset?
If we do a basis expansion on the features space we could get a linear classifier which achieve zero prediction error.