Title: Watch: Choosing the problem type first
Course: Advanced Data Science: A Graduate Course > Section: Module 4: Supervised learning: regression and classification

Watch

▶ Recorded lecture (~30 min) + slides ↓:
- Choose the problem type first: a continuous target points to regression, a label or a probability points to classification, and no target at all points back to the unsupervised methods of the previous module. The framing decision constrains everything downstream.
- Every model family carries a bias: linear models assume an additive, roughly linear signal and pay for violations with underfitting; single trees have low bias but high variance; ensembles trade a little interpretability to average that variance away.
- Interpretation comes bundled with the fit. A linear coefficient is a change in the expected outcome per unit of a predictor, holding others fixed; a logistic coefficient is a change in log-odds, and its exponent is an odds ratio. If you cannot say the sentence, you have not finished the model.
- Trees beat linear models when the true signal has interactions and nonlinearities that you have not hand-engineered, and when predictors are on wildly different scales; linear models win when the signal is genuinely additive, data are scarce, or you need transparent, monotone effects.
- The bias–variance trade-off is not abstract: it is why the training error keeps dropping while the held-out error turns back up, and why model selection must happen on data the model never touched.
- Validation is the real deliverable. A number without a leakage-safe split behind it is decoration, and the rest of the module is built to make that lesson expensive to ignore.

“Advanced Data Science by Michelle Blomberg is licensed under CC BY-NC-SA 4.0.”