Hello All, looking at the Video on Regularization I finally do not really understand interest of Ridge if regularisation is basically embedded in linear regression (/classification) scikit models…
To be more specific : what is the interest of using for classification models RidgeClassifier versus LogisticRegression? I tested some results and found differences (but also because looks like solver choices are not the same…).
Thanks a lot & ghave a nice day.
Both models apply the same L2 penalty on the weights. They are equivalent in this regard. What is changing is the loss that the models minimize.
With ridge, the mean squared error is minimized while in the logistic regression case, the log-loss is minimized. Indeed, the strategy to adapt the original regression problem into a classification problem is tackled differently.
In ridge, the classification targets are scale to [-1, 1] before applying the ridge regression loss while in with the logistic regression, the sigmoid function is used to squash the classification targets (and prediction) between [0, 1].
So since the losses are different, the solutions will differ as well.
OK thanks clear
Does the classification targets should be regression targets since it is for ridge regression?
We were discussing RidgeClassifier and not Ridge. So RidgeClassifier is a modification of Ridge to deal with classification problem.