How to get the coef in correct way?
Hi,
I think you have forgotten to change the value of your columns argument in the dataframe.
Just before the question 14 (because you have to deal with all features when previously you dealed with 12 features only) this code is provided :
preprocessor.fit(data)
feature_names = (preprocessor.named_transformers_["onehotencoder"]
.get_feature_names(categorical_columns)).tolist()
feature_names += numerical_columns
So the correct defintion of your dataframe should be :
coefs = pd.DataFrame(coef, columns=feature_names)
“onehotencoder” give a KeyError because you surely called your categorical_preprocessor with another name in your ColumnTransformer to define your preprocessor. Just change by the name you used and that should be fine.
Yes, you called it "cat" from what I can see from the diagram
Hi Abdelrahman,
I noticed in the previous screen copy that you did not fit the preprocessor (no number beetween the [ ] in the line of the notebook). If you do not fit your preprocessor I do not think you can have an attribute get_feature_names.
Do your preprocessor.fit(data) before trying to get the feature_names.
P.S.: you do not need to repeat your question for each people you want a help from since the forum send a message to each participants of a thread
In addition, here it seems that you are using a pipeline. So you need to access the OneHotEncoder from the Pipeline. If you check the code of each of the previous answer, you will probably get how we access the OneHotEncoder.
I tried to mimic your code from the data you shown to us :
from sklearn import set_config
set_config(display='diagram')
import pandas as pd
adult_census = pd.read_csv("../datasets/adult-census.csv")
target = adult_census["class"]
data = adult_census.drop(columns=["class", "education-num"])
from sklearn.compose import make_column_selector as selector
numerical_columns_selector = selector(dtype_exclude=object)
categorical_columns_selector = selector(dtype_include=object)
num_columns = numerical_columns_selector(data)
cat_columns = categorical_columns_selector(data)
from sklearn.preprocessing import StandardScaler, OneHotEncoder
categorical_preprocessor = OneHotEncoder(handle_unknown="ignore")
numerical_preprocessor = StandardScaler()
from sklearn.compose import ColumnTransformer
preprocessor = ColumnTransformer([
('cat', categorical_preprocessor, cat_columns),
('num', numerical_preprocessor, num_columns)])
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
model = make_pipeline(preprocessor, LogisticRegression(max_iter=500))
preprocessor.fit(data)
feature_names = (preprocessor.named_transformers_["cat"]
.get_feature_names(cat_columns)).tolist()
feature_names += num_columns
feature_names
['workclass_ ?',
'workclass_ Federal-gov',
'workclass_ Local-gov',
'workclass_ Never-worked',
'workclass_ Private',
'workclass_ Self-emp-inc',
'workclass_ Self-emp-not-inc',
'workclass_ State-gov',
'workclass_ Without-pay',
'education_ 10th',
'education_ 11th',
'education_ 12th',
'education_ 1st-4th',
'education_ 5th-6th',
'education_ 7th-8th',
'education_ 9th',
'education_ Assoc-acdm',
'education_ Assoc-voc',
'education_ Bachelors',
'education_ Doctorate',
'education_ HS-grad',
'education_ Masters',
'education_ Preschool',
'education_ Prof-school',
'education_ Some-college',
'marital-status_ Divorced',
'marital-status_ Married-AF-spouse',
'marital-status_ Married-civ-spouse',
'marital-status_ Married-spouse-absent',
'marital-status_ Never-married',
'marital-status_ Separated',
'marital-status_ Widowed',
'occupation_ ?',
'occupation_ Adm-clerical',
'occupation_ Armed-Forces',
'occupation_ Craft-repair',
'occupation_ Exec-managerial',
'occupation_ Farming-fishing',
'occupation_ Handlers-cleaners',
'occupation_ Machine-op-inspct',
'occupation_ Other-service',
'occupation_ Priv-house-serv',
'occupation_ Prof-specialty',
'occupation_ Protective-serv',
'occupation_ Sales',
'occupation_ Tech-support',
'occupation_ Transport-moving',
'relationship_ Husband',
'relationship_ Not-in-family',
'relationship_ Other-relative',
'relationship_ Own-child',
'relationship_ Unmarried',
'relationship_ Wife',
'race_ Amer-Indian-Eskimo',
'race_ Asian-Pac-Islander',
'race_ Black',
'race_ Other',
'race_ White',
'sex_ Female',
'sex_ Male',
'native-country_ ?',
'native-country_ Cambodia',
'native-country_ Canada',
'native-country_ China',
'native-country_ Columbia',
'native-country_ Cuba',
'native-country_ Dominican-Republic',
'native-country_ Ecuador',
'native-country_ El-Salvador',
'native-country_ England',
'native-country_ France',
'native-country_ Germany',
'native-country_ Greece',
'native-country_ Guatemala',
'native-country_ Haiti',
'native-country_ Holand-Netherlands',
'native-country_ Honduras',
'native-country_ Hong',
'native-country_ Hungary',
'native-country_ India',
'native-country_ Iran',
'native-country_ Ireland',
'native-country_ Italy',
'native-country_ Jamaica',
'native-country_ Japan',
'native-country_ Laos',
'native-country_ Mexico',
'native-country_ Nicaragua',
'native-country_ Outlying-US(Guam-USVI-etc)',
'native-country_ Peru',
'native-country_ Philippines',
'native-country_ Poland',
'native-country_ Portugal',
'native-country_ Puerto-Rico',
'native-country_ Scotland',
'native-country_ South',
'native-country_ Taiwan',
'native-country_ Thailand',
'native-country_ Trinadad&Tobago',
'native-country_ United-States',
'native-country_ Vietnam',
'native-country_ Yugoslavia',
'age',
'capital-gain',
'capital-loss',
'hours-per-week']
And I had no problem
1 Like