Title: Practice and formative deliverable
Course: Advanced Data Science: A Graduate Course > Section: Module 7: Data and AI ethics and governance

Practice and formative deliverable

The practice items below are formative and ungraded. They build intuition before the graded assessment. The formative deliverable is checked for completion and returned with feedback, and it earns completion points on submission, credited on completion rather than scored for correctness.

These four interactive explorables build the intuition you will formalize in the assessment, and then you run a real disparate-impact check on the module’s case classifier, the people-affecting model under review.

1. Open PAIR’s Measuring Fairness, and move the decision threshold to see how improving one group’s outcome degrades a competing fairness metric; note the exact tradeoff you cannot escape.
2. Work through MLU-Explain’s Equality of Odds, to see how equalized true-positive and false-positive rates differ from parity and calibration.
3. Play Survival of the Best Fit, a hiring-bias game that shows how historical data launders past discrimination into an automated system.
4. Audit a model with Aequitas, to see how a bias-and-fairness toolkit reports subgroup disparities against a reference group.
5. Probe predictions with Google’s What-If Tool, to inspect counterfactuals and per-subgroup performance without writing new code.

◎ Formative deliverable: Run a disparate-impact check on the module’s case classifier and its dataset. Choose a sensitive attribute in the data, compute one fairness metric (for example selection rate or false-positive rate) for each subgroup, and report the disparate-impact ratio of the least-favored to the most-favored group. In two or three sentences, name which competing metric your chosen one trades against and in which direction. Checked for completion and for a correctly computed ratio; the instructor posts a short model answer and flags any group whose sample is too small to interpret.

“Advanced Data Science by Michelle Blomberg is licensed under CC BY-NC-SA 4.0.”