Dial Your Course

Course Design · Case study

A browser-resident course review instrument: a faculty member drops in a Canvas course export, nineteen checks run against it, and the approved fixes come back as a package ready to reimport.

Version

0.2

Status

Version 1 in use

Owner

Michelle Blomberg

1. Summary

Online course preparation contains a substantial deterministic component that is currently performed by reading, and therefore performed inconsistently: alternative text, heading structure, contrast, deadline distribution, competency coverage, workload against the credit contract, and the presence of mandated language. These are checkable properties of a file.

Dial Your Course parses an IMS Common Cartridge export (.imscc), computes nineteen independent checks from the parsed structure, reports each finding with the figures that produced it, and, on the instructor’s approval, rewrites the package and returns it for reimport. All computation occurs in the browser. Nothing is transmitted.

Version 1 is built and working. It runs in any browser and inside Canvas today, with no server, no account, and no district action required, and it is in use in the Digital Media Arts program. The multi-faculty pilot in section 12 has not started.

A second problem compounds the first. The Canvas rich content editor sanitizes HTML on save, discarding <style> blocks, external stylesheets, and class-based CSS. Only inline style attributes survive. The instrument writes styling inline for that reason, so a designed, accessible template arrives intact.

The instrument reports, it does not certify. A check that cannot compute does not fire. No finding is emitted without the figures behind it, and no standard is reported as satisfied unless it was verified. Twenty-three of the fifty OSCQR standards concern conduct during the term or require a judgment of quality; those are enumerated with the established method for evaluating each, and are not scored.

2. Goal

Reduce the mechanical component of online course review to a single pass over the course package, and return a remediated package rather than a checklist.

The driver is seat time. The Higher Learning Commission has flagged seat hours in online courses at this institution. A credit carries a fixed allocation of student learning hours and the syllabus commits to them, but in an online course there is no classroom to observe, so the course package is the only evidence the hours were delivered. It is also the artefact a reviewer will read.

Three questions follow from that, and the instrument answers all three. Does the work fit the hours the credit allows? Are the objectives written in verbs a student can be observed performing? Is every outcome the course claims actually evidenced by something it grades? Assessment validity follows from the third: which graded items could a model complete unaided, and what process evidence does the rubric already collect?

The instrument did not begin with a rubric. It began with the checks its author performed by hand, cycle after cycle, as a peer and lead reviewer for Quality Matters and OSCQR. It automates the countable part of that review and declines to automate the part that takes judgment.

3. Users and context

User	Context	What the instrument gives them
Faculty	Preparing an online course for delivery or for formal review. No technical fluency assumed.	A private rehearsal for review, run on demand, on their own machine. Findings tied to the standard and the location in the course, and a corrected package returned.
Instructional designers	Supporting many courses across a term, with the countable checks consuming the hours that consultation should have.	The mechanical pass done before the conversation starts, so their time goes to design judgment rather than to counting alt attributes.
Departments and programs	Multiple sections, multiple instructors, one approved course record. Preparing for program review.	Consistency across sections, and outcome coverage that can be shown rather than asserted.
Reviewers	Formal Quality Matters or OSCQR review, once a year for most courses.	A course that arrives already correct on everything establishable mechanically, so review time is spent where professional judgment is required.

Formal review is thorough, valuable, and for many faculty exposing, because the first close reading a course receives is frequently the one that evaluates it. Small and entirely fixable problems (a missing alternative text attribute, an outcome never attached to a rubric, a module whose workload has drifted past the credit contract) surface during review rather than before it. This instrument is a rehearsal, not a substitute. It reports to the instructor and to no one else: no score is submitted, no record is retained, no administrator receives the result.

4. How it works

Export the course from Canvas, drop the file in, choose the checks. The parser constructs a course object from the cartridge, and every check is a pure function of that object. Each finding names the standard it comes from and where in the course to fix it. The instructor approves the changes, the instrument writes them back and repackages the cartridge for reimport, and it prints a short list of the few things only Canvas itself can do.

Does the course hold together

Canvas setup: outcome objects, rubric bindings, assignment groups, and the weighting scheme, reported as one task with a sequence rather than as separate defects.
Outcome to assessment alignment, presented as an alignment matrix.
Course competencies resolved to the assignment and criterion that assess them.
Measurable objectives (OSCQR 9), tested against a verb bank, with observable replacements at the appropriate level of Bloom’s Taxonomy.
Assessment validity under generative AI, and the AI resistance audit.
Seat hours and workload against the credit contract.

Can a student use it

Universal Design for Learning (the CAST framework, distinguished from accessibility, with which it is routinely conflated).
Course overview and information (OSCQR 1 to 10), and technology and support (11 to 15).
Design and layout (16 to 28): alternative text, descriptive link text, table header cells, typeface.
Content and activities (29 to 37): captioning and PDF accessibility.
Interaction (38 to 44), and assessment and feedback (45 to 50).

Remediate and reimport

Visual consistency: counts distinct typefaces, sizes, and colours, applies a selected palette inline, and verifies contrast against WCAG AA at selection, so a failing pair cannot be applied.
Due dates: flags dates carrying more than two deadlines, and graded items with no due date, which appear in neither the student To Do list nor the calendar.
Student data scan of course Files, which flags and never deletes.
Export round-trip.

Not scored, enumerated for human review

Ten standards cannot be verified from a file, including Regular and Substantive Interaction (34 CFR 600.2) and the think-aloud protocol (Ericsson and Simon, 1984). Each is presented with the reason a parser cannot evaluate it and the established method for doing so, with a checkbox retained in the browser. The default is unchecked, because nothing has yet verified them.

Seat hours

Total student hours equal credits multiplied by forty. Load hours alter how that total is split between direct instruction and outside work; they never alter the total. The figure that matters to an instructor is the outside-work allowance, because that is the portion the course package spends: the reading, the video, and the graded work. It sits on the approved course record, is not recoverable from the export, and is therefore requested from the instructor.

The instrument estimates hours for every graded item and every page of reading and video, reconciles the aggregate against the allowance at module and course level, makes every estimate correctable, and writes the confirmed figure into each module overview, which satisfies OSCQR Standard 2. A student who knows what a module costs plans for it.

An estimate is not evidence. Video duration is assumed, because the cartridge stores links rather than durations, and effort per graded item is inferred from point weight, which is a proxy. A wrong number in a module header is worse than no number, because students plan around it. The remedy is not a better estimator but measurement: one question at submission, answered by the people who did the work, converts the published figure from a guess into a claim with data behind it.

Assessment validity and the AI resistance audit

Two checks operate here and they are deliberately distinct. The first evaluates each graded item against two conditions: whether a generative model could produce a passing response within sixty seconds, and whether the item carries consequential weight, defined as at or above the median point value of the graded items in the course. Only the conjunction constitutes a finding. A low-stakes formative check is rehearsal, not measurement, and a model completing it costs the course nothing. Where an item fails both conditions, the finding is framed as a threat to construct validity rather than as academic integrity, and a substitution at equivalent cognitive demand is proposed.

The second inverts the question. It reads the rubric criteria of every graded item, not the assignment titles, and reports which of seven kinds of process evidence each one already collects: physical capture, process artefacts, critique given, revision from a named critique, articulated rationale, situated content, and file-level evidence. Each measured assessment is banded (strong, adequate, exposed) and returned with two or three specific additions. Low-stakes items are listed and explicitly not judged.

Authenticity of task does not confer resistance to generative substitution. A model will write a portfolio case study, a client rationale, and a reflective essay, all of which are authentic tasks. What resists substitution is the record of making: iteration and drafting, documented response to critique demonstrably received, situated local context, and live defence of a decision. The artefact never protects the assessment. The record of its making does. The audit therefore measures what a rubric collects, not what an assignment asks for.

Accessibility, and the boundary of what a parser can see

The instrument verifies the structural properties recoverable from the package: alternative text on every image (OSCQR 36), descriptive link text (37), header cells on every table (24 to 26), typeface (23), and captions required on video (35). It does not open PDFs and does not claim to. It counts them and routes the instructor to YuJa Panorama, which the district already licenses and which scores files in place, performs text recognition, tags headings and reading order, and generates alternative formats. A check that reimplements a tool the institution already owns adds no capability and one more thing to maintain.

Two boundaries are stated rather than papered over. OSCQR 23 (typeface) is SUNY’s editorial standard, not an accessibility requirement: neither WCAG nor Section 508 specifies a typeface family, so the instrument reports the departure without characterising it as an accessibility defect. And caption accuracy is not verified by automated checking anywhere, including here. Automated checkers confirm that a caption track exists. Automatic speech recognition runs at roughly 70 to 90 percent accuracy and degrades most on domain terminology, which in a technical course is frequently the content being taught. An unedited track can satisfy a compliance check while remaining difficult to use for the student who depends on it.

Requirements and acceptance criteria

Each requirement is one behavior. The checks beside it are what a builder or tester confirms before that requirement is called done. Status reflects what this document records as built.

#	Area	Requirement	Done when	Status
R1	How it works: intake and parsing	The user can drop a Canvas Common Cartridge export (.imscc) into the page and choose which checks to run.	A tester drops a valid .imscc file onto the page and sees the list of checks available to select. The tester selects a subset of checks and the report contains findings from the selected checks only. The tester runs the tool on exports with outcomes absent, outcomes present and unbound, and outcomes bound, and a report is produced in all three cases.	Built
R2	How it works: findings	The product reports every finding with the standard it comes from, the location in the course where it is fixed, and the figures that produced it.	Every finding in the report shows a standard reference, a course location and the counts or values behind it. A check whose input is missing from the package produces no finding and no pass result for that check. No standard is shown as satisfied unless a check verified it in the package.	Built
R3	Does the course hold together	The product reports Canvas setup (outcome objects, rubric bindings, assignment groups and the weighting scheme) as one task with an ordered sequence of steps.	On a course with several setup gaps, the report shows one Canvas setup task and not separate defects. The task lists its steps in a stated order covering outcomes, rubric bindings, assignment groups and weighting.	Built
R4	Does the course hold together	The product presents outcome to assessment alignment as a matrix and resolves each course competency to the assignment and rubric criterion that assess it.	On an export with bound outcomes, the report shows a matrix of outcomes against graded items. Each competency is listed with the assignment and criterion that assess it, or is flagged as having none. On an export with no outcomes file, the alignment check does not fire and the seat hours and objectives checks still run.	Built
R5	Does the course hold together	The product tests each objective against a verb bank and proposes an observable replacement verb at the appropriate level of Bloom’s Taxonomy.	An objective using a verb outside the bank is flagged against OSCQR 9. Each flagged objective is shown with at least one proposed replacement verb and its taxonomy level. An objective that already uses an observable verb from the bank is not flagged.	Built
R6	Seat hours	The product estimates student hours for the course content, reconciles them against the outside-work allowance entered by the instructor, and writes the confirmed figure into each module overview.	The tester is asked for the outside-work allowance and the tool shows total student hours as credits multiplied by forty. The report shows an hours estimate for every graded item, reading page and video, with totals compared to the allowance at module level and course level. The tester changes an estimate and the module and course totals update. After approval and export, each module overview in the package contains the confirmed hours figure.	Built
R7	Assessment validity and the AI resistance audit	The product flags a graded item as a construct validity finding only when it is both completable by a generative model within sixty seconds and at or above the median point value of graded items in the course.	An item that meets both conditions appears as a finding worded as a threat to construct validity, not as academic integrity. An item that meets only one of the two conditions does not appear as a finding. The finding shows the item point value and the course median used for the comparison.	Built
R8	Assessment validity and the AI resistance audit	The product reads the rubric criteria of every graded item and reports which of the seven kinds of process evidence each one collects, with a band of strong, adequate or exposed.	Each measured assessment is listed with the kinds of process evidence found in its rubric criteria and one of the three bands. Each measured assessment is returned with two or three specific additions. Low-stakes items are listed separately and carry no band. Changing an assignment title without changing its rubric does not change the result.	Built
R9	Can a student use it	The product checks the package against the OSCQR groups for course overview and information, technology and support, interaction, and assessment and feedback.	The report contains a section for each of the four groups, with each finding naming its OSCQR standard number. Course information that is absent from the Canvas pages in the export is reported as not found. A standard in these groups that cannot be verified from the file appears in the human review list and not as a pass.	Built
R10	Accessibility, and the boundary of what a parser can see	The product verifies alternative text on images, descriptive link text, header cells on tables, typeface, and the presence of captions on video, and counts PDF files without opening them.	A package containing an image without alternative text, a link with non-descriptive text and a table without header cells produces one finding for each, with its location. A typeface departure is reported against OSCQR 23 and is not labelled an accessibility defect. The report states the number of PDF files and directs the instructor to YuJa Panorama for them. The report states that caption accuracy was not verified.	Built
R11	Remediate and reimport	The product counts the distinct typefaces, sizes and colours in the course and applies a palette chosen by the instructor as inline styles, refusing any colour pair that fails WCAG AA contrast.	The report shows the number of distinct typefaces, sizes and colours found. The tester selects a palette with a failing colour pair and the tool does not allow it to be applied. After a passing palette is applied and exported, the page content carries inline style attributes and no style block or class-based styling.	Built
R12	Remediate and reimport	The product flags any date that carries more than two deadlines and any graded item that has no due date.	A course with three items due on one date shows a finding naming that date and the items. A graded item with no due date is flagged, with a note that it appears in neither the student To Do list nor the calendar. A date with two deadlines is not flagged.	Built
R13	Remediate and reimport	The product scans the filenames and file types in course Files, flags candidates that may hold student data, and never deletes a file.	A package whose Files area contains a gradebook spreadsheet or roster shows that file as a flagged candidate. After export, every flagged file is still present in the package. The scan result is based on filename and type only, and the report says so.	Built
R14	Remediate and reimport	The product writes only the changes the instructor approved into the package, repackages it for reimport, and lists the tasks that must be done by hand in Canvas.	A change the tester did not approve is absent from the exported package. The exported package has the same file count as the original and all XML files are byte-identical. The exported package imports into Canvas and every approved change is visible after the editor saves the page. After export, the tool shows a short list of the remaining tasks that only Canvas can perform.	Built
R15	Not scored, enumerated for human review	The product lists each standard that cannot be verified from a file, with the reason a parser cannot evaluate it, the established method for evaluating it, and a checkbox that is unchecked by default.	Each listed standard shows a reason, a method and a checkbox, and none of them carries a score. On first use every checkbox is unchecked. The tester checks a box, closes the browser, reopens the tool on the same machine and sees the box still checked.	Built
R16	Data, privacy, and governance	The product processes the course package entirely in the browser tab and stores nothing except the human review checkboxes.	With the browser network panel open during a full run and export, no request carries course content. The tool runs with no sign-in and no account. After a run, browser storage for the page holds the checkbox states and no course content.	Built
R17	Build and portability	The product is delivered as one static HTML file whose only external dependency is JSZip, and it runs from disk, from static web hosting, and from a Canvas course as a linked or embedded file.	The file opened from disk completes a full run and export. The file uploaded to Canvas course Files and linked or embedded in a page completes a full run with instructor rights only. With JSZip placed beside the file and the script tag repointed, the tool completes a full run with no request to a third-party script source. A tester can reach and operate every disclosure control by keyboard, and text contrast meets WCAG AA.	Built
R18	Version 2: paste bridge	The product composes a prompt that contains only the flagged passages, and the user pastes it into the sanctioned model and pastes the reply back into the tool.	The composed prompt contains the flagged assignment prompt and its rubric criteria and no other course content. With the network panel open, no request to a model originates from the page. The pasted reply appears as an editable draft that the tester can approve, edit or discard. A discarded draft makes no change to the exported package.	Version 2 preview, not working
R19	Version 2: redesign suggestions	The product proposes two or three substitute assessments for each flagged item, at equivalent cognitive demand, chosen from the reference library by finding type and by the evidence the rubric criteria show.	Each flagged item shows two or three proposed substitutions. Each substitution can be traced to an entry in the reference library. The same package run twice selects the same reference entries for the same finding. No proposal is described as an essay a model cannot write or as a detector.	Version 2 preview, not working
R20	Version 2: reference library	The product holds its reference library as versioned files beside the tool and never calls a scholarly index while running.	The library files are present beside the tool and carry a version. With the network panel open during a run, no request goes to an external index. Each library entry carries a citation.	Version 2 preview, not working
R21	Version 2: Universal Design for Learning	The product flags content offered in a single modality, assignments that accept one submission format, and modules with no learner choice, self-assessment or activation of prior knowledge, and proposes a specific addition for each.	A module whose content exists in one modality is flagged with a proposed second modality. An assignment accepting one submission format is flagged with a proposed alternative means of expression. The report states how Universal Design for Learning differs from accessibility.	Version 2 preview, not working
R22	Version 2: RSI writer	The product generates an editable interaction plan and Regular and Substantive Interaction statement and checks the draft against the OSCQR interaction standards and the two-of-five substantive interaction test.	The draft contains a weekly teaching announcement, a feedback schedule, a facilitated discussion, a response-time commitment and a monitoring and outreach policy. The tester can edit the draft before copying it. The tool shows which of the five substantive interaction activities the draft covers and whether at least two are met.	Version 2 preview, not working
R23	Version 2: instructional-design lens	The user can select an instructional design model and see the same findings reworded through that model.	Selecting a lens changes the wording or grouping of the report. The number and identity of findings are the same before and after a lens is selected. Clearing the selection returns the report to its default form.	Version 2 preview, not working
R24	Version 2: Builder mode	The user can enter course objectives, lesson objectives and a topic, and the product returns a starter course package to import.	With only objectives and a topic entered, and no file loaded, the tool returns a module outline with a stated time cost. The scaffold shows an assessment matched to each objective, an interaction plan and Universal Design for Learning options. The module time costs sum to no more than the seat hour budget. The starter package imports into Canvas.	Version 2 preview, not working
R25	Version 2: one tool, four front doors, and export formats	The product presents Audit, Builder, Syllabus and Report as tabs over one tool, and the user can export the report as standalone HTML, as PDF and as a Word file in addition to the Canvas cartridge.	The four tabs are present and each opens its named function. A seat hour figure confirmed in one tab appears in the Syllabus tab without retyping. The exported HTML report opens offline with its styling intact. The Canvas cartridge export remains available and unchanged.	Version 2 preview, not working
R26	Version 2: institutional proxy	The product can send flagged passages to a server-side endpoint that holds the institutional credential and authenticates the user through single sign-on.	The page source and browser storage contain no credential. A user who is not signed in through single sign-on receives no generated text. The request body contains flagged passages only, and the response contains generated text only. Changing the model requires a change to one endpoint and one request shape, with no change to the checks.	Designed, not built

5. Data, privacy, and governance

A course export is instructor content, not student content. Canvas intentionally excludes enrollments, grades, submissions, quiz attempts, and student-authored discussion replies from a course export, and Instructure omits them for FERPA compliance. The cartridge carries course content only, so the instrument is not meant to receive student records. Course Files can still hold them, a gradebook spreadsheet or a roster for example, so the tool flags those files for the instructor to remove.

Nothing is transmitted. The cartridge is opened, read, rewritten, and repackaged inside the browser tab. There is no server, no account, no vendor intermediary, and no network request carrying course content, and the claim is verifiable in the network panel during execution. The privacy position is architectural rather than contractual: there is no processor to contract with.

The only thing stored is the set of human-review checkboxes, in localStorage, on the reviewer’s own machine, so a review can span sessions. No course content is retained.

Residual risk in course Files

Course Files travels with every export and every course copy, and is published by default. Instructors accumulate gradebook spreadsheets, rosters, and letters of reference there. The instrument scans filenames and types, flags candidates, and does not delete, because removing an instructor’s material without instruction is a worse defect than the exposure it addresses.

Security review question	Response
Where is course data transmitted?	Nowhere. All processing occurs in the browser. Verifiable in the network panel.
Is student data processed?	No. A Canvas export excludes enrollments, grades, and submissions by design.
Is authentication required?	No. There is no account. The only stored state is the review checkboxes on the reviewer’s own machine.
What is retained?	Human-review checkboxes in localStorage on the reviewer’s machine. No course content.
Third-party dependencies?	JSZip, MIT licensed, loaded from a CDN and vendorable beside the file.
Accessibility of the instrument itself?	Semantic markup, keyboard-operable disclosure controls, AA contrast throughout.

6. Build and portability

A single static HTML document. Vanilla JavaScript, no framework, no build step, no server-side component, no database, no runtime configuration, no authentication. The one external dependency is JSZip, loaded from cdnjs.cloudflare.com; where an institutional Content Security Policy prohibits third-party script sources, JSZip is vendored beside the HTML file and the script tag repointed. That is the only modification any hosting environment is expected to require. Deployment consists of placing one file where a browser can reach it, which is what makes district-scale hosting a question of a web request rather than of budget: no per-seat license, no inference bill.

Built with a model, shipped without one

Both claims are true and they are separate. The instrument was designed and written in sustained collaboration with a language model: the Canvas cartridge schema was reverse-engineered against real exports that way, and the checks were drafted, adversarially tested, and corrected that way. What ships contains no call to a language model, no credential, and no request that carries course content. The one outside request loads the zip library. Every finding is produced by deterministic parsing and a decision table. Where the report proposes a remediation, that proposal is a rule with an unmet condition, not a generated sentence.

This is a design decision with three consequences a generative implementation could not offer. It cannot hallucinate a finding or a standard. It cannot leak, because nothing is transmitted. It costs nothing to run. The deterministic checks are also better for this class of problem: verifying that mandated language appears verbatim is a string comparison, and a model performs worse at it because it accepts paraphrase as equivalence.

The cost is symmetrical and should not be obscured. The decision table cannot notice an outcome its author did not consider, and it will propose an outcome for a course that merely uses a trigger word in passing. A model would perform better at that and considerably worse at inventing nothing. For an instrument whose only asset is that a faculty member believes its findings, that is the correct trade, and it remains a trade.

Is the model swappable

Today there is no model to swap: version 1 is deterministic end to end. Version 2 surfaces five capabilities the instrument already detects and cannot resolve without generation (draft alternative text, rewritten objectives, a situated assessment substitution, caption accuracy, and prose clarity), and in the current build they run through the paste bridge described below.

The generative layer is specified as model-agnostic. It sends only flagged passages, never the course package, and it receives text that arrives as an editable draft subject to the same approval as any other change. Any chat-completion endpoint satisfies that contract: Claude, Google Gemini on Vertex AI, or a district-hosted model behind an internal endpoint. Swapping the model means changing one endpoint and one request shape, not the checks, because the diagnostic work is deterministic and stays that way.

One constraint governs all of it: a credential cannot reside in the client. The instrument is a static document served to the browser, so any embedded credential is a published credential. This is a property of the delivery model and is not mitigable by obfuscation. Three options exist.

Bridge, with no infrastructure

The instrument composes a prompt containing only the flagged passages. The instructor pastes it into the model the institution has already sanctioned and pastes the reply back. No credential, no server, and no request originates from the page, so only what the instructor personally pastes leaves the machine. It is inelegant, it is available immediately, and it is the recommendation for the pilot.

Instructor-supplied key

Each instructor obtains a developer key and stores it in their own browser, and the instrument calls the model API directly. This does not scale: each instructor must obtain and fund a key, and a key held in a client application is a recognised exposure class that institutional IT is unlikely to sanction. Not recommended.

Institutional proxy

A minimal server-side endpoint holds one institutional credential (a service account or a Workload Identity binding), authenticates instructors through existing single sign-on, forwards the flagged passages, and returns only the generated text. No credential reaches the browser. This is the only path that scales past a pilot, and it requires a cloud project, an endpoint, and a data-handling position. It is the institutional ask.

What the institution would provide for the proxy: a sanctioned endpoint reachable from a proxy the college controls (Vertex AI if the district has a Google Cloud project; a developer API is a prototyping surface, not an enterprise control plane, and Gemini in Google Workspace is an end-user surface that exposes no programmable endpoint, which is a common source of confusion in procurement); a server-side credential, rotated on the standard schedule, never in the client; and a data-handling position covering flagged passages only. That last point is what keeps the request document-scale rather than corpus-scale, the inference cost negligible, and the review tractable.

Could it run on district infrastructure, or inside Canvas

Stage	Method	Requires	Suitable for
Local	The file is opened from disk	Nothing	Individual evaluation
Present hosting	GitHub Pages, the current arrangement	Nothing	Pilot with named colleagues
Canvas	Uploaded to course Files and linked, or embedded in a page	Instructor rights only	Departmental use without an IT request
College web server	A static file under a college path	Web team, one file	College-wide availability under an institutional URL
District	Static hosting under a district domain	District IT, security review	District-wide adoption

It runs inside Canvas today, as a linked or embedded static file, with no LTI registration and no district action. An LTI placement would only change how it is launched, not what it does, because it keeps nothing on a server and needs no roster. Moving to district infrastructure means copying one file and, if a Content Security Policy applies, vendoring JSZip.

Dependencies

What has to be in place, and who provides it.

Needed	Provided by	Needed for
A Canvas course export in .imscc format	Instructor	R1 and every check
The outside-work hours allowance from the approved course record	Instructor, from the Curriculum Office record	R6 seat hours
Competency text from the district course record, compared by hand because the record cannot be retrieved by script	Instructor, from the Curriculum Office record	R4 competency coverage
JSZip library, from a public script host or vendored beside the file	Product owner	R1, R14, R17 and the hosting milestones
YuJa Panorama licence for PDF accessibility	District	R10 PDF referral
Static hosting on GitHub Pages	Product owner	R17 and the pilot
Completion of the two outcomes export tests (cache reset, then rubric binding)	Product owner	R4 on courses whose outcomes were imported by CSV, and R24 reading objectives from a package
A campus role with account-level Outcomes rights to create a program outcome group	Canvas administration, role to be named	R4 program outcome alignment
Manual rebinding of account-level outcomes in Canvas after reimport	Instructor	R14 round-trip
Three to five faculty across two disciplines and their instructional designers, each with one live online course	Digital Media Arts department and instructional design staff	Departmental pilot
Receipt of pilot data and agreement to offer the tool in course preparation support	Teaching and Learning Centre	Rollout after the pilot
A place for one static file under a college path	College web team	College hosting milestone
Security review and static hosting under a district domain	District IT	District hosting milestone
A generative model the institution has sanctioned for instructor use	Institution	R18 paste bridge
A cloud project, a sanctioned model endpoint, a rotated server-side credential, single sign-on and a data-handling position for flagged passages	District IT	R26 institutional proxy
A vetted reference library, refreshed at build time from the OpenAlex and ERIC indexes	Product owner	R19 and R20
The existing syllabus checker, currently a separate Apps Script tool	Product owner	R25 Syllabus tab

7. Version 2, from detection to redesign

Version 1 diagnoses and stops. The assessment validity audit flags which graded items a model could complete unaided and which rubrics already collect a record of the student’s thinking, then hands the finding to the faculty member. It does not say what to do about it. Version 2 answers that question: for a flagged item, it proposes a more authentic version of the assessment, and the faculty member decides whether to use it. The instrument moves from telling a faculty member their assignment is exposed to offering them a redesign that is not.

The premise is the one the audit already rests on. When a model can produce the artefact, the artefact stops being evidence of who produced it. That is a validity problem, not an academic-integrity problem, and it is not solved by a harder prompt or a detector. It is solved by changing what the assessment collects, moving the evidence from the finished artefact to the record of making it, at the same level of cognitive demand the original asked for.

How it works with the AI model

Version 2 is the generative layer specified in section 6, and it is deliberately model-agnostic. For each flagged item, the instrument sends only the flagged passage, the assignment prompt and its rubric criteria, to a chat-completion endpoint. It never sends the course package, and it never sends student data, because none is present. The model returns a proposed redesign as an editable draft, which the faculty member approves, edits, or discards under the same approval gate as every other change the instrument makes. Nothing is rewritten without instruction.

The endpoint is a contract, not a vendor. Claude, Google Gemini on Vertex AI, or a district-hosted model all satisfy it, so moving from the Claude the tool was built on to the Gemini the district is standardising on is one endpoint and one request-shape change, with the deterministic checks untouched. The one hard rule is that a credential cannot sit in a static page served to the browser: for the pilot the instrument composes the prompt and the faculty member pastes it into the model the institution has already sanctioned (no credential, no server), and to scale, a minimal institutional proxy holds one server-side credential behind single sign-on. The detection stays deterministic and free; only the suggestion calls a model.

The suggestions are discipline-aware

Authentic process evidence is not the same in every field, so the redesign a flagged item receives depends on what making the work actually looks like. A studio critique leaves a trail of iteration and response to feedback, a lab leaves a notebook, a programming task leaves a commit history, and a writing task leaves drafts, sources, and decisions. Version 2 reads the rubric criteria and proposes the evidence that fits the discipline rather than one generic fix. The hardest case, and the one worth working through, is the one every program has: the paper.

The hard case, English 101 and the essay

A first-year composition course is built on the essay, and the essay is precisely what a generative model produces best. The artefact cannot carry the assessment, and no amount of prompt engineering makes an essay a model cannot write. Version 2 does not pretend otherwise. It does not propose an “AI-proof” essay or a detector. It proposes moving part of the grade onto evidence a model cannot generate on the student’s behalf, so the paper stops being the only thing assessed.

Anchor the prompt to something only this student encountered. A specific class discussion, a local source the student found and annotated, a personal artefact, a text read in this section this week. A model does not know what the student saw, so it cannot ground a response in it.
Collect the record of making. An outline committed before drafting, the revision history, an annotated bibliography in the student’s own words, and a short paragraph on what they cut and why. This is the trail a finished essay does not contain.
Add a brief defence. Three questions generated from the student’s own specific claims, with follow-ups. A student who did not do the thinking cannot sustain “why not the other reading?” on work they did not do.
Grade the disclosure, not the absence of AI. Ask the student to mark what the model produced and what they wrote, and to revise the model’s draft against a named critique. Grade the delineation and the revision. This is the only move that assesses the student who used AI well, which is the student the course actually wants to graduate.

The boundary is plain: none of this makes a paper impossible to write with a model, and that is the point rather than a failure of it. The goal is not an unbeatable artefact, it is an assessment where the artefact is no longer the only evidence, so the grade rests on a reasoning trail the student can show and defend. Making that shift assessable without surveillance, by asking for the process as a deliverable rather than monitoring the student, is the whole design problem, and it is the same one every writing program in the country is now facing.

The alternate-assessment menu

The English 101 case is one path through a general menu. Version 2 carries a categorised bank of authentic, generation-resistant substitutions drawn from the current two-lane consensus: a secured lane that validates individual achievement, in person or oral and supervised, and an open lane where AI use is scaffolded and the process, not the product, is graded. For a flagged item the instrument proposes two or three substitutions at equivalent cognitive demand, selected by what making the work leaves behind: process and portfolio evidence, iterative drafts against a named critique, an oral defence or walkthrough, a supervised in-class performance, a prompt localised to what only this student encountered, an applied project with a real client or constraint, reflection tied to a specific revision, applied identification in place of a recall quiz, or disclosure-graded AI use where the student marks what the model produced and revises it against a critique. Each is grounded in the reference library rather than generated from nothing.

Universal Design for Learning, as recommendations rather than a label

Version 1 names UDL and delivers structural accessibility, which is not the same thing. Accessibility asks whether a student can reach the content; UDL asks whether there is more than one way to engage with it, perceive it, and act on it. Version 2 makes UDL checkable against the CAST 3.0 guidelines and their three principles, engagement, representation, and action and expression. It flags content that exists in a single modality and proposes a second, an assignment that accepts one submission format and proposes an alternative means of expression, and a module with no learner choice, no self-assessment, or no activation of prior knowledge, and it offers the specific addition rather than a citation to the framework. The distinction from accessibility is stated in the report, because the two are routinely conflated.

The RSI writer

Regular and Substantive Interaction cannot be read from a package, which is why version 1 enumerates it and stops. It is also the federal line (34 CFR 600.2) that separates distance education from correspondence and protects Title IV eligibility, so an empty result is a poor place to leave a faculty member. Version 2 turns the dead end into a deliverable. It generates a drop-in interaction plan and RSI statement the instructor edits and pastes into the syllabus and course: a weekly instructor-authored teaching announcement, a stated feedback schedule, an instructor-facilitated discussion, a published response-time commitment, and a monitoring-and-outreach policy that uses Canvas analytics by hand and collects no student data. The draft is checked against the OSCQR interaction standards and the two-of-five substantive-interaction test the regulation requires, direct instruction, feedback on coursework, answering content questions, facilitating discussion, or an institution-approved activity.

The instructional-design lens

The same findings can be read through the model a faculty member was trained in. A checkbox reframes the report through a chosen lens without changing what was found. Backward Design asks whether every objective has evidence and every graded item serves an objective, which is the instrument’s native logic. ADDIE surfaces the missing evaluation loop. Merrill’s First Principles flags a module that jumps to application with no worked example or real problem. Gagne’s nine events read as a per-module checklist. Dick and Carey emphasise where one component has drifted out of alignment with another. Connectivism and personal learning environments, the author’s own theoretical floor, value real audiences, networked tools, and a public artefact, and flag a course that is a closed loop. The instrument does not impose a model; it offers the vocabulary the instructor already thinks in, and a course does not use every model at once.

Builder mode

Everything above audits a course that exists. Builder mode scaffolds one that does not. It borrows the pattern from Wayfinder: the page does the structured work, and the model, through the same bridge, does the open-ended reasoning. The instructor states the course objectives, the lesson objectives, and the topic, and the builder returns a starting scaffold at the right level of Bloom’s Taxonomy: a module outline with a stated time cost inside the seat-hour budget, an assessment matched to each objective at equivalent cognitive demand, an RSI plan, and UDL options for engagement and expression. It produces a starter package to import and, where a suggestion needs generation, a prompt to hand to the sanctioned model. The ask is deliberately small, objectives and a topic, so the useful first output arrives before any file exists.

Why the builder takes objectives by hand

Builder mode asks for the objectives directly rather than reading them from a package, and that is a decision the export forced. On the test course, verbed outcomes visible in Canvas were absent from the export, reproduced twice, with the outcomes file not written at all rather than written empty. The leading explanation is a cached has_outcomes flag on the course record that the CSV importer never clears, and a separate confirmed defect makes account-level outcomes serialise as references that cannot resolve on reimport. Both are documented in the outcomes-export research (section 11) and both are unfixable from a package. Two cheap tests are queued to learn whether the drop is universal or particular to that one course: re-save the course settings to bust the cache and export again, and failing that, bind one outcome to a rubric criterion and export again. Until they resolve, any feature that needs objectives, the builder most of all, asks the instructor for them rather than trusting the export to carry them. If the tests show a normally built course does export its outcomes, the builder can offer to read them; the manual path stays as the fallback either way.

One tool, four front doors

Version 2 presents as tabs over one instrument that shares one reference library and one approval gate. Audit is the version 1 checks. Builder scaffolds from objectives. Syllabus brings in the syllabus checker, currently a separate Apps Script tool, so the required-language, seat-hour, and RSI checks sit beside the course audit. Report is the findings, exportable. Rolling the syllabus checker in removes a second tool to learn and lets the seat-hour figure and the RSI statement move between the syllabus and the course without retyping.

Export in more than one format

Version 1 exports one thing, a Canvas cartridge for reimport. Version 2 keeps that and adds report and rebuilt-course exports that need no server: a standalone HTML report, which is the page serialised with its styles inlined and satisfies the rule against linking a downloadable document; a print-to-PDF stylesheet with an optional one-click PDF; and a Word export for an instructor who wants a rebuilt course or a report offline. The cartridge stays the primary course export; the rest are for reading and sharing.

The reference library

The suggestions are only as good as what grounds them, and a model left to invent them produces plausible, generic, and occasionally wrong advice, which is exactly what an instrument whose whole asset is trust cannot afford. Version 2 grounds every suggestion in a curated reference library held beside the tool as versioned files: the authentic-assessment menu, the CAST 3.0 UDL guidelines, the RSI practices and the regulation, the instructional-design models, and an OSCQR-to-Quality-Matters crosswalk. Retrieval is deterministic, the finding type selects the reference block, and the model writes only the last sentence against material it did not invent, the same line the tool already draws between a decision table and a generated phrase. The library is authored and kept current at build time, not at run time: a build-time pass queries the open scholarly indexes, OpenAlex and ERIC, both free and keyless, the author vets what comes back, and the vetted claim and its citation go into the library. The running tool never calls those indexes, so there is no latency, no rate limit, and no unvetted text in a trust-critical path. Gemini Notebook is a useful workbench for that authoring; it is not the tool’s engine, because it exposes no stable endpoint to build against.

Status: version 2 is in development, with faculty testing next. Nothing in this section is in use yet. The build, audit, preview, and action-plan views, the reference library, and the authentic-assessment suggestions all run today, and the authentic-assessment piece has produced results. Generative features run through the paste bridge described in section 6, not a direct connection, because a credential cannot sit in a static page. Still ahead: a round-trip test of the starter package in Canvas, and a wider faculty review of whether the suggestions read as credible to the people who teach the subject. The line holds throughout: detection stays deterministic and free, the model writes only last-mile suggestions against the grounded library, every suggestion is an editable draft under the same approval gate, and no finding, binding, or outcome is ever fabricated.

8. Pilot and testing

The pilot takes routine checking work off faculty and instructional designers. It does not replace either. Every finding goes to the instructor, who decides; the instrument revises nothing without instruction, and the judgments that require professional expertise are not automated at all, they are enumerated with the method for evaluating each and handed back.

Element	Plan
Who	Three to five faculty across two disciplines, starting in Digital Media Arts, plus the instructional designers who support those courses.
What	One live online course each, run end to end: export, check, set the seat-time estimates, approve, export the corrected package, reimport, and verify every change landed.
Period	One term, followed by departmental adoption across all Digital Media Arts sections for a second term.
Measured	Every finding known to be false, recorded by severity. Which findings were acted upon and which were dismissed. Whether the corrected package reimports cleanly. How long the pass took against the same review done by hand. What the instructor had to correct in the workload estimate.
Reported	To the Teaching and Learning Centre, with pilot data rather than a demonstration.

A false positive on a high-severity finding is the failure mode that destroys confidence in an instrument of this kind, and surfacing it is the pilot’s purpose. A finding that is universally dismissed is either incorrect or badly worded, and both are defects.

9. Definition of success

Version 1.0 means the pilot passed and the instrument is cleared for rollout. The thresholds are stated in advance so the decision is not made by impression.

Threshold	Condition to clear
Correctness	No false positive on a high-severity finding survives the pilot uncorrected. Any finding a pilot participant can demonstrate to be false is fixed or withdrawn before rollout.
Round-trip integrity	Every corrected package reimports into Canvas with the file count preserved, all XML byte-identical, and no styling discarded by the editor.
Usefulness	Each participant acts on findings they would not otherwise have caught, and no check is dismissed by every participant. A check dismissed by all is rewritten or removed.
Seat hours	Each pilot course publishes a confirmed outside-work figure per module, corrected by the instructor, and at least one course closes the loop by asking students at submission whether the published budget matched the work.
Trust	Participants report they would run it again unprompted, on a course of their own, without being asked.

No figure above is claimed. They are the conditions under which the pilot would become a 1.0.

10. Rollout

Departmental adoption. Digital Media Arts, all sections, one term, with the findings log kept.
Present to the Teaching and Learning Centre with pilot data, and hand them the instrument to offer as part of course preparation support.
College hosting. One static file, one request to the web team, JSZip vendored in anticipation of an institutional Content Security Policy.
District hosting under a district domain, following a security review. The questions that review will ask are answered in section 5.
Generative layer, if the pilot supports it. Bridge first, at zero cost and zero risk. The institutional proxy is requested only with pilot evidence behind it, never in advance of any.

Explicitly out of scope

Capability	Why it is out of scope
Batch analysis across many courses	Reports to an institution rather than to an instructor. That is a different instrument, with a different consent model.
Modality equivalence, an online section against its in-person master	Requires two exports and an authoritative course master. Feasible, and a reasonable later version, but it answers an institutional question rather than a self-check.
Reporting to an accreditation or assessment office	The instrument’s usefulness depends on faculty trusting that it reports to them alone. Routing results elsewhere would end that.
Student engagement monitoring	Requires continuous student data, which this tool is not designed to handle. Canvas New Analytics already serves that need.

These are recorded because they are the predictable next requests. Declining them is a design decision rather than an oversight.

Out of scope

Batch analysis across many courses, because that reports to an institution and not to an instructor.
Comparison of an online section against its in-person master course.
Reporting results to an accreditation office, an assessment office or any administrator.
Student engagement monitoring or any feature that needs continuous student data.
Certifying a course or submitting a score; the product reports to the instructor only.
Opening or remediating PDF files; the product counts them and refers the instructor to the tool the district already licenses.
Verifying caption accuracy; the product confirms only that captions are present.
Deleting instructor files or making any change to the package without instructor approval.

11. Open questions and risks

Outcomes do not reliably survive an .imscc export. This is a known Canvas limitation and a real constraint on rollout. On the test course, five outcome groups and fourteen verbed outcomes are visible in Canvas, imported by Outcomes CSV. The export contains none of them, and course_settings/learning_outcomes.xml is absent rather than empty. Reproduced on two separate exports. Until outcomes reach the export, the outcome-alignment half of the instrument cannot run on that course. Seat time and objectives still run, because assignments do export.

The leading explanation is a cached has_outcomes? boolean keyed on the course record: the CSV importer creates the outcomes without touching the course row, a stale value persists, and no file is written at all, which matches the symptom exactly. Two tests are queued, cheapest first: re-save course settings to bust the cache and export again; failing that, bind one outcome to a rubric criterion and export again. A separate, confirmed defect will bite on reimport regardless: account-level outcomes (the institutional learning outcomes) serialize as references, not copies, so on import the identifier cannot resolve and the criterion arrives as a plain criterion with no outcome attached. No tool can fix that from a package. It is two minutes each, by hand, in Canvas, which is what the post-export to-do list exists for.

Open question or risk	Priority	Notes
Does any campus role hold account-level Outcomes rights?	High	This single fact determines whether program outcomes are a two-week problem or a two-year one. Program outcomes are currently absent from Canvas at every level, and entering them by hand per course would create one unrelated object per course, which Canvas cannot aggregate. The unblock is one administrative action: create a program outcome group at the account or sub-account level, after which Find Outcome works for program outcomes exactly as it does for institutional ones.
Workload is an estimate, not a measurement	High	Video duration is assumed, effort is inferred from point weight. The figure is a starting position for a curriculum conversation, and it becomes evidence only when students confirm it at submission.
Assessment validity is a pattern match	Medium	It identifies items exhibiting the structure of a model-completable assessment. It does not establish that one is.
Course information is detected by keyword	Medium	Where that information lives in Simple Syllabus rather than in a Canvas page, it will not appear in the export and the finding will be wrong.
The district course record cannot be retrieved	Medium	The Curriculum Office does not serve its pages to scripts from another origin, so competency text must be compared by the instructor. No browser-resident tool can do otherwise.
Content Security Policy	Low	JSZip is loaded from a CDN. Vendoring it beside the file is a one-line change and should be done before any institutional hosting request.

Questions to settle before the next build

What are the nineteen checks by name, and what is the exact pass or fail rule for each, including the OSCQR group checks that section 4 names without rules?
Which version 2 features exist in the preview today, and which of them are in scope for the next build?
Which changes does version 1 write into the package on approval, and how does that list fit the rule that all XML stays byte-identical?
Which rules produce the seat hour estimate: the hours per page of reading, the assumed video duration, and the conversion from point weight to hours?
Is the human review list ten standards or twenty-three, and which standards are on it?

12. Roadmap

Milestone	Target	Notes
Parser and nineteen checks	Built	Operational against Canvas Common Cartridge exports, across all package states, including outcomes absent, present and unbound, and bound.
Export round-trip	Built	File count preserved, all XML byte-identical, output reduced to ASCII with numeric entities, no <style> block emitted, package reimports.
Sample report and style guide	Built	Published alongside the instrument.
Outcomes export tests	Next	Cache-bust test, then the rubric-binding test. Determines whether the alignment check can run on a course whose outcomes came in by CSV.
Departmental pilot	Planned, term not set	Three to five faculty, two disciplines, one term. Every false finding recorded by severity.
Vendored JSZip	Before hosting	In anticipation of an institutional Content Security Policy.
College hosting	Post-pilot	One static file, one request to the web team.
Generative layer, version 2	Post-pilot	Bridge first. Institutional proxy requested with pilot evidence behind it. Five capabilities, all detected today and disabled: draft alternative text, rewritten objectives, situated assessment substitutions, caption accuracy, prose clarity.
District hosting	Post-pilot	Following security review.

OSCQR, the SUNY Online Course Quality Review Rubric, is made available under a Creative Commons Attribution 4.0 International License by the State University of New York. This instrument applies the rubric and is not affiliated with or endorsed by SUNY.