AI-assisted course design
Course Design · Case study
Recent research, not my default way of working. I design and build courses by hand; this is me testing whether a panel of AI agents can draft from vetted open educational resources, whether a reusable skill can hold the standards, and whether a course-builder tool can check a course against a quality bar. I am measuring how far the approach actually holds up.
Goal
To establish that a rigorous, standards-compliant online course can be produced through orchestrated AI agents while holding instructional quality to recognized standards, and to subject that claim to the most demanding version of the test. The demonstration deliberately builds in disciplines outside the designer’s own domain expertise as a methodological control: with content knowledge held constant, the quality of a resulting course is attributable to the method and its embedded learning science rather than to subject-matter command. If the course withstands scrutiny, the method is what produced it, and the method is what transfers.
Audience
The immediate audience is the instructor who inherits a course with little lead time, often an adjunct assigned a section shortly before the term begins, and who needs a defensible starting point rather than a blank shell. More broadly, the work addresses learning designers, instructional-design teams, and centers for teaching and learning adopting AI-assisted course development, together with the academic leaders responsible for determining whether such development can be conducted responsibly and to standard.
Process
The method runs as an orchestrated agent panel. A subject-matter-expert agent and an instructional-designer agent draft from license-verified open educational resources using backward design and constructive alignment; a synthetic-student agent stress-tests each draft against the incoming learner; and every pass is held to the encoded standards, OSCQR, Regular and Substantive Interaction, Universal Design for Learning, and WCAG 2.1 AA. The course-builder tool then checks the result against the quality bar, and a named faculty subject-matter expert verifies the content before anything reaches a student.
Technology
- The build methodology is packaged as a reusable Claude skill (the equivalent construct in Google’s Gemini environment is a Gem) that orchestrates the agent panel described on The Build Skill tab.
- A course-grounded AI tutor, built as a public Gemini Notebook and linked at the top of the course, answers student questions from the course and its vetted open resources, with citations, and stays inside the material rather than the open web.
- Open educational resources and interactive, parameter-driven explorables, each license-verified: MLU-Explain, TensorFlow Playground, R2D3, Setosa, Distill, and Google’s PAIR fairness visualizations.
- Eight simulation-based summative assessments, one per module, with a live stakeholder defense. Four are built as playable simulations, plus a defense practice tool. All are engineered to resist generative substitution, supported by the Synthetic Data Vault, Faker, and Evidently (drift detection).
- Eight instructor slide decks with speaker-note scripts and a complete instructor preparation kit.
- Delivered as static HTML and CSS.
Outcomes
A reviewed draft and a working method. The data science course has not been verified by the faculty member who would teach it, and it has not been tested with or delivered to students.
This course was an experiment: how far does AI get when no subject-matter expert is in the room? I do not recommend building a course that way. After it was built, an outside reviewer with a master’s in applied statistics and data science estimated the course was about 78 percent usable as built; her findings are on the Applied Data Science page.