Title: Assessment: Recover the Signal
Course: Advanced Data Science: A Graduate Course > Section: Module 2: Preprocessing messy and dynamic data

Assessment: Recover the Signal

Each learner receives a personalized messy dataset. The instructor holds privately a clean data-generating process with known parameters; from it, a per-student seed produces a corrupted copy carrying MNAR missingness (values are withheld in a way that depends on the withheld value itself, so naive deletion biases your estimates), unit inconsistencies (a subset of records reported in different units without labels), duplicate records (some exact, some near-duplicates differing by a keyed field), and outliers of a known distribution (a defined fraction drawn from a contaminating distribution, alongside genuine extremes from the true process). Your task is to recover the underlying signal. Cleaning quality is scored numerically by how closely your recovered summary statistics (means, variances, selected quantiles, and a target correlation) match the hidden truth the instructor holds. Because the truth and the answer key are private and unique to your seed, the grade rewards a principled process, not a lucky guess or a borrowed solution.

What you will do, step by step:

1. Profile the corruption: inspect distributions, dtypes, ranges, and null patterns; use missingno to map where and how values are absent before touching anything.
2. Diagnose the missingness mechanism: gather evidence for MCAR versus MAR versus MNAR (co-occurrence of nulls, dependence of missingness on observed covariates, and telltale truncation suggesting the missing value itself drives its absence), and state your conclusion with its support.
3. Reconcile units: detect the inconsistently-scaled subset, infer the conversion, and bring all records onto a common unit, documenting the detection rule.
4. Resolve duplicates: define exact-match and near-match criteria, decide which record survives a near-duplicate collision, and justify the rule.
5. Handle outliers with a defended rule: apply an outlier test from the NIST handbook or a stated robust criterion, then decide per case whether each flagged point is an error to correct/remove or a genuine extreme to retain, and record why.
6. Impute appropriately: choose a strategy whose assumptions match your diagnosed mechanism (for MNAR, this typically means modeling the mechanism or adding a missingness indicator rather than mean-filling), fitting any learned imputer without leaking information across partitions.
7. Document every step as reversible: keep the raw file immutable, transform copies, and log each transformation so the raw state can be reconstructed.
8. Assemble the steps into a single reproducible pipeline that runs end-to-end from raw file to recovered statistics.
9. Report the recovered statistics together with a confidence statement: state which recovered quantities you trust most and least, and why, given the mechanisms you diagnosed.

What to submit: a reproducible cleaning pipeline (a script or notebook that runs from the raw file to the reported numbers on a fresh machine); a decision log defending each choice with its mechanism assumption and reversibility; and the recovered summary statistics with your confidence statement.

How it is graded:

Criterion | What strong work shows | Pts
Numeric closeness to hidden truth | Recovered means, variances, quantiles, and target correlation fall within tight tolerance of the instructor’s private values across the seeded dataset. | 17
Correct mechanism diagnosis | The missingness is correctly identified as MNAR with cited evidence, and the diagnosis drives the chosen treatment rather than being stated and ignored. | 17
Defended decisions | Every unit fix, duplicate rule, outlier ruling, and imputation choice is justified by an explicit assumption, not by reflex; outliers are correctly triaged into errors versus genuine extremes. | 17
Reversibility and documentation | Raw data is preserved immutable; each transformation is logged with enough detail to reconstruct the raw state and re-derive the step. | 17
Reproducibility | The pipeline runs end-to-end from the raw file to the reported statistics on a clean environment, with no leakage of information across partitions. | 16
Confidence statement | The report honestly characterizes which recovered quantities are more and less trustworthy and ties that judgment to the diagnosed mechanisms. | 16
Total | 100

◆ Why this resists generative substitution: The clean data-generating process and its answer key are private to the instructor and the dataset is per-student-seeded, so no correct target exists in any public corpus. A pasted generic cleaning solution cannot match a truth it has never seen; a high score requires diagnosing your specific corruption and defending choices against your hidden parameters, which is exactly the process the numeric grade measures.

“Advanced Data Science by Michelle Blomberg is licensed under CC BY-NC-SA 4.0.”