Interpretation of One Hot Encoded features

In the solution to the exercises it says ‘Dropping one of the one-hot encoded column is a common practice, especially for binary categorical features. Note however that this breaks symmetry between categories and impacts the number of coefficients of the model, their values, and thus their meaning, especially when applying strong regularization.’ for One Hot Encoding features. Can you please clarify on how to interpret the features in such a scenario?

I see that we did not see this question. Sorry about that.

I am not sure what you mean by “such scenario”. If you mean in the scenario where you drop a feature nevertheless and with a strong regularization, then we don’t have a correspondence between a category and a weight. So this is difficult to provide a meaning :slight_smile:

The value is a multiplicative factor to a specific category given that at the same time there is one category dropped. When regularization comes into the game, we don’t anymore give a weight that minimizes the error and therefore an association between X and y but also a value that should be as minimum as possible. So in terms of interpretability, this is quite complex to come to a conclusion regarding the values apart from comparing the absolute value of the different coefficients.