01 Chapter

Supervised Learning: Regression & Classification

The workhorse algorithms of applied ML — ranked by practical usage.

Supervised learning covers the workhorse algorithms of applied ML — models that learn from labeled examples to predict a target. Below are the most popular families, ranked by practical usage, with what they’re best for and where they’re commonly applied.

For tabular business data, start with LightGBM/CatBoost/XGBoost, Random Forest, and Logistic/Linear Regression.
Gradient boosting is widely recognized as a leading practical method for structured prediction and has been prominent in ML competitions.

#	Algorithm	Best for	Common fields
1	Gradient Boosted Trees: XGBoost, LightGBM, CatBoost, GBM	High-performing tabular prediction	Finance insurance fraud detection pricing marketing medicine churn prediction
2	Random Forest	Strong baseline, robust tabular modeling	Healthcare credit risk customer analytics ecology operations
3	Logistic Regression	Binary/multiclass classification, interpretable baseline	Credit scoring medical risk marketing response A/B testing
4	Linear Regression / Ridge / Lasso / Elastic Net	Numeric prediction, interpretable models	Economics real estate sales forecasting demand forecasting
5	Decision Tree	Simple interpretable rules	Business rules education healthcare triage explainable ML
6	Support Vector Machine / SVR	Smaller datasets, high-dimensional data	Bioinformatics text classification image features
7	k-Nearest Neighbors	Simple similarity-based prediction	Recommenders anomaly detection small classification tasks
8	Naive Bayes	Fast text classification	Spam detection sentiment analysis document classification
9	Generalized Linear Models	Structured statistical modeling	Biostatistics actuarial science econometrics
10	Neural Networks / MLPs	Nonlinear supervised learning	Tabular DL forecasting embeddings scientific ML