HistGradientBoostingRegressor and XGBoost and

Module 6. Ensemble of models Ensemble based on boosting
#1

Hi,
When studying this topic I came across XGBoost, which seems to be used in industry. Is it related to HistGradientBoostingRegressor and if yes, do you know how they compare in terms of performance?
Just curious.
Thank you,
Edgar

#2

The short answer is that XGBoost with approximated histograms is very similar to HistGradientBoostingRegressor/HistGradientBoostingClassifier and should have similar performance.

There is some scikit-learn activity on making systematic benchmarks between XGBoost, LightGBM and scikit-learn but this is still work in progress. The current outcome shows that they have similar performance.

You can look at some difference between different gradient boosting algorithms if you wish: XGBoost, LightGBM or CatBoost — which boosting algorithm should I use? | by Aviv Nahon | Riskified Technology | Medium.

Note that scikit-learn implementation is closer to the LightGBM one.

#3

Thank you very much for the helpful info and the link!
Edgar