User Testing Plan: How to Test CopaMigo with Real Students

01. What We Are Testing (and Not Testing)

What we are testing

• Whether students understand what CopaMigo is and how to use it within the first 30 seconds of seeing it
• Whether the empathy-first response feels right or feels canned and AI-generated
• Whether the routing makes sense (does the right office actually show up for the question?)
• Whether the contact info on each card is enough to act on
• Whether the tool works correctly on a phone (not just a laptop)
• Whether the voice-to-text input works for the way students actually talk
• Whether the tool feels safe to use for sensitive topics (mental health, basic needs, sexual misconduct)
• Whether students would actually use it again, and recommend it to a friend

What we are NOT testing

• Whether the tool is “good enough” in some absolute sense, comparison to other tools is for a separate evaluation, not user testing
• Whether the AI is technically accurate on every possible question, the goal is connection, not omniscience
• The student’s intelligence, vocabulary, or technology skills, if they got confused, that’s a tool problem, not a student problem

02. Who to Recruit

The right pool is eight to ten students with deliberate variety. More is fine; fewer is too few to see patterns. Recruit a mix across these dimensions:

• Year: at least two first-semester students, two continuing students, one returning student (someone who took time off and came back)
• Language: at least two Spanish-dominant or bilingual students (Spanish is the second-largest language at GCC and the multilingual design promise needs real pressure-testing, not a single token data point). Other languages are bonus if available (Vietnamese, Arabic, Mandarin, Tagalog).
• Program: a mix of academic and CTE programs. Don’t only recruit from your own department
• Identity: at least one veteran or military-connected student, one student parent, one student who has used counseling or basic-needs services
• Accessibility (required, not optional): at least one student who uses assistive technology to navigate web tools day-to-day. The most important categories to cover are screen reader users (VoiceOver on iOS/Mac, TalkBack on Android, NVDA or JAWS on Windows), students who are blind or have low vision, students with limited mobility who use voice control or switch input, and students with cognitive disabilities for whom plain language and clear structure matter most. The DRS office can help recruit; some students self-identify and others won’t, so frame the ask broadly.
• Neurodivergent (required): at least one autistic student. Autistic students are a significant portion of the GCC population and surface testing concerns that other testers won’t, including literal interpretation of language (idioms and metaphors in the empathy message can land wrong), sensitivity to sensory and interaction surprises (modal pop-ups, animations, unexpected audio playback), and a preference for predictability (a tool whose behavior shifts depending on context can feel unstable). One autistic tester will catch issues that ten neurotypical testers miss. ADHD students are also valuable; their concerns overlap with what Scenario 5 already targets, but pulling them in deliberately rather than incidentally produces better feedback.
• Tech comfort: at least one student who is not particularly tech-confident; at least one who is fluent and skeptical of AI

03. Before the Session

The day before

• Confirm the student is coming and remind them of the time
• Decide where: a quiet room, not a hallway. The student needs to feel comfortable saying hard things
• Have two devices ready: a laptop and a phone with the tool open. Some students will want to use one, some the other
• Have a recording sheet ready for each scenario (or open a Google Doc with the template from Section 06)
• Confirm the prototype is working, open it yourself and verify the API key is loaded and the tool responds

Right before the session starts

• Greet the student. Thank them. Make sure they have water if they want it
• Read the participant introduction (Section 04) out loud. Don’t paraphrase, the framing matters
• Confirm consent to record observations (you are recording your observations, not their face or voice unless they explicitly agree)
• Open the tool to the welcome screen. Hand them the device they prefer

04. What to Read to the Participant

05. Test Scenarios

Session length: 50–60 minutes total. That breaks down as: 5 min intro, 40–45 min for scenarios, 5–10 min debrief. To stay in that window, run the 6 Core Scenarios every session, they take about 35–40 minutes, then pick one Spotlight Set (2–3 scenarios on a specific topic) to go deeper on, based on what you most need to learn that day. Rotate which Spotlight Set you use across sessions so you cover everything over time without burning any single participant.

For each scenario: hand the student a printed card with just the situation on it, no department name, no hint about where to go. Watch what they type and what they make of the response. The exact words they type are data.

Core Scenarios, run every session (6 scenarios, ~35–40 min)

These six cover the highest-traffic routing situations and the most common failure modes. Together they test empathy tone, multi-service routing, language switching, advising vs. counseling confusion, and crisis triage.

Optional Swap-Ins, replace a Core scenario if relevant (~5 min each)

Use these to replace one Core scenario when you have a specific student profile, for example, swap in Scenario 9 if several participants came straight from high school, or Scenario 10 if you want to see how a first-time user explores the tool.

Spotlight A, Financial Aid (pick 2, ~12 min)

Financial aid questions are the highest-stakes situations CopaMigo handles. Run this Spotlight when you want to verify the SAP, R2T4, and drop/withdrawal knowledge. Pick two, a confident student and a panicked one give different data.

Financial aid questions are among the highest-stakes situations a student brings to CopaMigo. The bot now has detailed knowledge of Pell, SAP, W grades, refund rules, and withdrawals, but that knowledge is only useful if it lands in plain language and doesn’t scare the student. These scenarios test whether the tool gives a useful first answer AND still gets the student to a human.

Spotlight B, Tech, Printing & Files (pick 2, ~10 min)

These test the Library vs. IT routing, the most common mismatch in the tool. Scenario 18 (lost file) is especially valuable because it has emotional weight that surfaces whether the empathy message lands before the practical answer.

Spotlight C, AI Policy & New Cards (pick 2, ~10 min)

AI policy questions will increase in fall. Scenario 21 tests the most common integrity question. Scenario 22 is the most emotionally charged. Scenarios 27 and 28 should be run as a pair, 27 tests whether the bot offers to help write a prompt, 28 tests whether it actually delivers one when the student takes the offer. This two-turn sequence tests a unique Tier 1 direct-answer behavior that exists nowhere else in CopaMigo.

Spotlight D, DRS & Veterans (pick 2, ~10 min)

Higher emotional weight, run this Spotlight with a moderator who’s comfortable with sensitive topics. Scenario 23 (IEP) works well with any first-semester student. Scenarios 24 and 26 need participants who self-identify as Deaf or veteran-connected respectively.

06. Recording the Session

Use a recording sheet for each scenario. The template below works on paper, in a Google Doc, or in a spreadsheet, whichever the moderator prefers.

Per-scenario recording template

What to specifically check on each scenario

These are the technical checks that should pass for every scenario, regardless of which scenario it is:

• URLs are clickable. Every URL on every card should be a real link, not text. Tap each one and confirm it opens.
• Phone numbers are tappable on mobile. If you’re on a phone, tapping a phone number should open the dialer with the number filled in. If it doesn’t, that’s a bug.
• Map links open the right page. Every “Map to [building]” link should open the building’s GCC location page, not 404 or wrong building.
• Hours are present and look current. Every card should show hours. They should match what’s on the official GCC page for that office.
• Voice input transcribes correctly. When the student uses voice, what they said should appear in the input box accurately. Listen for words that got mangled.
• Voice output speaks the response back. If the student used voice, the tool should read the response aloud. The voice should match the language of the input.
• The page renders properly on mobile. No text cut off, no buttons unreachable, no horizontal scroll.

What to check during accessibility-focused sessions

These checks apply when the tester is using assistive technology. Run them in addition to the general checks above, not instead of them.

• Screen reader announces every interactive element. Tab through the page with a screen reader on. The mic button, send button, reset button, every service card link, and every chat fallback should announce themselves with a meaningful label, not just “button” or “link.”
• Focus order is logical. Tab order should follow the visual order of the page: welcome message, then input area, then response cards, then contact info inside cards, then chat fallback. If focus jumps around or skips elements, that’s a bug.
• Focus is visible. When the tester tabs to an element, there should be a visible focus indicator (outline, color change, etc.). If you can’t tell where focus is, neither can a sighted keyboard user.
• Cards and modals are reachable by keyboard. The “Still stuck?” chat button on each card should be reachable by tab. The prototype chat modal should trap focus inside it (so Tab cycles within the modal, not behind it) and close on Escape.
• Voice input still works with screen reader running. Some screen readers conflict with the Web Speech API. Confirm voice input doesn’t break when VoiceOver, TalkBack, or NVDA is active.
• Reading order matches visual hierarchy. A screen reader should hear the empathy message first, then the cards in order. The visual hierarchy should match the announced hierarchy.
• Color is not the only signal. The student should be able to distinguish primary contact info from the chat fallback link without relying on color alone. The chat fallback is intentionally muted visually, confirm it’s still distinguishable to a low-vision user, and not invisible.
• Text resizes to 200% without breaking the layout. Zoom the browser to 200%. The page should still be usable; nothing should get cut off, overlapped, or unreachable.
• For autistic testers specifically: watch for moments where the empathy message uses an idiom or metaphor that lands wrong, where a modal pop-up arrives suddenly without warning, or where the tool’s behavior changes in ways the tester didn’t expect. These won’t surface in a checklist; they surface in the tester’s reactions, so listen.

07. Post-Test Interview

After all scenarios, have a 5–10 minute conversation. Ask open questions. Don’t defend the tool or explain anything; just listen.

Questions to ask

• “What was your overall impression?”
• “What was the best moment using it? What was the worst?”
• “Was there a moment where the tool felt like it really understood what you were asking? Was there a moment where it felt off?”
• “If you had a real situation like one of these, would you actually use this tool? Why or why not?”
• “Would you tell a friend about it? In what situation would you tell them to try it?”
• “Was there anything that made you trust it more, or trust it less?”
• “Did you ever think ‘this is just an AI bot, this isn’t going to help’? When?”
• “What’s missing? What did you wish it could do?”
• “Anything you didn’t say that you want to say now?”

08. After the Session

Within 24 hours

• Write up the session notes while details are fresh. Quotes especially.
• Flag any bugs found (broken links, wrong routing, mobile issues) for immediate fix before the next session
• Note any patterns emerging across sessions if you’ve run multiple already

After all sessions are done

• Compile findings across all participants. Look for patterns: did multiple students hit the same confusion point?
• Sort findings into three buckets: critical bugs (fix immediately), design improvements (next iteration), out-of-scope (note for future)
• Pick out the three to five most striking student quotes for use in stakeholder presentations
• Draft a one-page summary: what was learned, what changed, what is still uncertain
• Share back with participants if they want, students appreciate knowing their feedback mattered

09. A Note on Sensitive Topics

Several scenarios in this plan involve mental health, sexual misconduct, food insecurity, or other situations that may resonate with the student you’re testing with. They are testing a hypothetical, but they may be carrying real things too.

• Watch for distress. If a student becomes visibly upset during a scenario, pause. Ask if they want to skip it or take a break. Their wellbeing comes first; the testing data does not.
• Have real resources at hand. Know the actual GCC counseling number (623-845-3064) and the 988 line. If the conversation moves from hypothetical to real, you can offer the real resource without making it strange.
• Don’t probe. If a student volunteers a personal experience, listen briefly and warmly, but do not ask follow-up questions about the personal experience itself. The test session is not therapy and you are not their counselor.
• Decompress at the end. Wrap up with something light. Thank them. Make sure they are leaving in a normal headspace, not a stirred-up one.