AI-assisted course design
Case study · AI Tools & Strategy · Learning Design
Recent research, not my default way of working. I design and build courses by hand; this is me testing whether a panel of AI agents (a subject-matter expert, an instructional designer, and a synthetic student modeled on the incoming learner) can draft from vetted open educational resources, whether a reusable skill can hold the standards, and whether a course-builder tool can check a course against a quality bar, with a human directing it and owning every decision. I am measuring how far the approach actually holds up.
Goal
To establish that a rigorous, standards-compliant online course can be produced through orchestrated AI agents while holding instructional quality to recognized standards, and to subject that claim to the most demanding version of the test. The demonstration deliberately builds in disciplines outside the designer’s own domain expertise as a methodological control: with content knowledge held constant, the quality of a resulting course is attributable to the method and its embedded learning science rather than to subject-matter command. If the course withstands scrutiny, the method is what produced it, and the method is what transfers.
Audience
The immediate audience is the instructor who inherits a course with little lead time, often an adjunct assigned a section shortly before the term begins, and who needs a defensible starting point rather than a blank shell. What the method produces for that instructor, whether a structured foundation to build on or a near-complete course requiring only local adaptation and subject-matter verification, is itself one of the questions this experiment sets out to test. More broadly, the work addresses learning designers, instructional-design teams, and centers for teaching and learning adopting AI-assisted course development, together with the academic leaders responsible for determining whether such development can be conducted responsibly and to standard. It also answers the principal concern raised within the field, that AI-assisted production places unreviewed instructional content before students, by retaining a named human subject-matter expert as the accountable authority prior to delivery.
Process
The method runs as an orchestrated agent panel with a human directing every decision. A subject-matter-expert agent and an instructional-designer agent draft from license-verified open educational resources using backward design and constructive alignment; a synthetic-student agent stress-tests each draft against the incoming learner; and every pass is held to the encoded standards, OSCQR, Regular and Substantive Interaction, Universal Design for Learning, and WCAG 2.1 AA. The course-builder tool then checks the result against the quality bar, and a named faculty subject-matter expert verifies the content before anything reaches a student. The agents draft and check; the human owns every decision.
Technology
- The build methodology is packaged as a reusable Claude skill (the equivalent construct in Google’s Gemini environment is a Gem) that orchestrates an agent panel: a subject-matter-expert agent and an instructional-designer agent (each a PhD-level specialist with a decade of experience at a top research university), together with a synthetic-student agent modeled on the incoming learner’s academic preparation, so drafts are stress-tested against the reader they are written for.
- The design logic is backward design (Wiggins & McTighe) and constructive alignment (Biggs); the quality and compliance frameworks are encoded into the skill and enforced on every pass: OSCQR, Regular and Substantive Interaction (34 CFR 600.2), Universal Design for Learning (CAST), and WCAG 2.1 AA.
- A course-grounded AI tutor, built as a public Google NotebookLM and linked at the top of the course, answers student questions from the course and its vetted open resources, with citations, and stays inside the material rather than the open web. NotebookLM was chosen because it grounds strictly in the sources it is given and shares as a chatbot anyone with a Google account can use.
- Open educational resources and interactive, parameter-driven explorables, each license-verified: MLU-Explain, TensorFlow Playground, R2D3, Setosa, Distill, and Google’s PAIR fairness visualizations.
- Eight simulation-based summative assessments with a live stakeholder defense, engineered to resist generative substitution, supported by the Synthetic Data Vault, Faker, and Evidently (drift detection).
- Eight instructor slide decks with speaker-note scripts and a complete instructor preparation kit.
- Delivered as static HTML and CSS.
Outcomes
The method produced a complete, standards-aligned draft of an entire course in a technical discipline outside the designer’s own expertise, used as a control, with eight simulation-based assessments, a course-grounded NotebookLM tutor, eight instructor slide decks, and a full preparation kit. What is under measurement is how far the approach holds: whether an agent panel drafting from vetted open resources can meet recognized quality and compliance standards, whether the reusable skill keeps those standards across builds, and whether the course-builder tool can reliably check a course against the bar. The field’s central objection, that AI-assisted production puts unreviewed content in front of students, is answered by keeping a named human expert accountable before delivery.
Status
A complete, standards-aligned draft and a working method, held at its final step by design. The last move is to loop in the faculty subject-matter expert who will teach it, to verify the technical content and make it their own. Keeping that human in the loop is the point of the method, not an afterthought, and by design nothing reaches a student until it happens.