Title: Watch: Cleaning as defensible decisions
Course: Advanced Data Science: A Graduate Course > Section: Module 2: Preprocessing messy and dynamic data

Watch

▶ Recorded lecture (~25 min) + slides ↓:
- Cleaning as a sequence of defensible decisions: why “I dropped the bad rows” is never an acceptable methods sentence, and what a defensible alternative reads like.
- MCAR, MAR, and MNAR defined precisely, with a worked example of each, and the central claim that the mechanism, not the appearance of the hole, dictates the honest fix.
- Why every transformation must be reversible and documented: keeping the raw data immutable, transforming only copies, and recording each step so the raw state can always be reconstructed.
- Outliers versus errors: how a physically impossible value (a negative age) differs from a rare but real extreme (a genuine high earner), and why the two demand different treatments.
- Leakage-safe feature engineering: fitting imputers, scalers, and encoders on the training partition only, and the classic mistake of imputing or standardizing before splitting.
- Assembling the pieces into a single reproducible pipeline object so that the same transformations, learned from training data alone, apply identically to new data.

“Advanced Data Science by Michelle Blomberg is licensed under CC BY-NC-SA 4.0.”