Title: Overview
Course: Advanced Data Science: A Graduate Course > Section: Module 4: Supervised learning: regression and classification

Module 4: Supervised learning: regression and classification

Overview

Most modeling mistakes are made before a model is ever fit: the wrong problem type is chosen, or the evaluation is set up so that a good score means nothing. This module puts problem framing first. You will decide whether a task is regression (predicting a continuous quantity), classification (predicting a discrete label or its probability), or neither, and only then reach for a method. We work through three model families whose assumptions and failure modes you should be able to state from memory: linear regression, logistic regression, and tree ensembles (bagged trees and random forests). Throughout, interpretation is treated as inseparable from the fit, coefficients, odds ratios, and feature importances are read in the units of the problem, not admired as outputs. The module serves outcomes O3 (select and justify an analytic method that fits the problem and the data) and O4 (fit, interpret, and validate predictive models while stating their assumptions).

By the end of this module

- Distinguish regression, classification, and clustering, and diagnose which one a stated business or research question actually requires.
- Select a method that fits both the problem type and the structure of the data (sample size, dimensionality, linearity, class balance), and defend the choice against at least one alternative.
- Fit and interpret linear and logistic regression, reading coefficients, odds ratios, and predicted probabilities in the units of the domain.
- Fit and interpret tree ensembles, explain why they often beat linear models on tabular data with interactions, and read their feature-importance output critically.
- State the core assumptions and failure modes of each model family and design a leakage-safe validation scheme that produces an honest estimate of out-of-sample performance.

Readings

- An Introduction to Statistical Learning (Python edition, ISLP) (free to read · link). Read Chapter 3 (linear regression, including multiple regression, interaction terms, and diagnostics), Chapter 4 (classification, focusing on logistic regression and the confusion-matrix vocabulary), and Sections 8.1–8.2 (decision trees, then bagging and random forests). Work the Python lab notebook at the end of each chapter, do not just read them; run the cells, then perturb one thing (drop a predictor, add an interaction, change the number of trees) and predict the effect before you execute.
- Harvard CS109A runnable notebooks (MIT · adapt). Use the regression, classification, and tree/ensemble notebooks as a second pass in a different codebase. Compare how CS109A structures a train/validation/test split and a cross-validation loop against the ISLP labs, and note where the two courses make different default choices. Use ONLY these URLs.

“Advanced Data Science by Michelle Blomberg is licensed under CC BY-NC-SA 4.0.”