Assessment Design Playbook

Assessment Design Playbook

How to Use This Playbook Each category names where the assessment is substitutable with current AI, then leads with AI-resistant moves that secure the grade, followed by AI-forward moves that build AI use into the learning, the technologies that support each, and where the secured lane and TILT pairing sit for that category. The goal is to certify learning, not to catch students or watch them fail.

Two design rules run through every category: •

No method is fully AI-proof: proctoring, oral defense, and live video all have known limits (deepfakes and AI voiceovers among them), and each carries a cost in faculty time, student access, and the flexibility the program promises online students. Treat these as last-resort moves reserved for the highest-stakes work, not the first reach.

•

Whenever a move asks students to verify, critique, or improve AI output, pair it with the rubric and several annotated examples of strong work shared in advance. This is the Criteria component of the Transparency in Learning and Teaching (TILT) framework (Winkelmes); models let students aim at the target and close the gap for students who do not already know the conventions, which is the equity gain TILT was built to deliver.

Pedagogical Grounding The category pages that follow all resolve to two shared reference points: the Two-Lane Model, which decides whether an assessment needs controlled conditions or belongs in the open, AI-permitted lane, and the TILT Framework, which states that lane's rules plainly to students.

Two-Lane

Controlled

Assessment

Open /AI-Permitted

TILT

Purpo s

e

Task

Criteria

Two-Lane Model of Assessment University of Sydney; Danny Liu and Adam Bridgeman, 2024–2025

Assessment now has to do two jobs at once: assure that students achieved the learning outcomes, and help students learn to work productively with AI. Every assessment is sorted into one of two lanes.

Lane assignment is a course-level, assessment-by-assessment decision, and a single task can combine both lanes—for example, an open project capped by a short controlled check.

LANE 1 —CONTROLLED (ASSURANCE) Conditions are controlled enough to assure the work reflects this student's own capability. Controlled does not mean AI-banned—AI may be restricted, banned, or deliberately permitted, depending on the outcome being assured.

Purpose: A defensible signal of genuine achievement.

LANE 2 — OPEN (LEARNING) Uncontrolled conditions where AI use is permitted and even encouraged.

Students demonstrate skills in the environment where they will actually use them, while building AI literacy—evaluating, refining, and taking responsibility for AI output rather than simply submitting it.

Purpose: Build AI-capable graduates.

Pairs with TILT: the two-lane model decides which lane a task lives in; TILT makes the rules of that lane explicit to students—especially the open lane, where “how may I use AI here, and how will it be graded?” must be stated plainly.

TILT Framework Transparency in Learning and Teaching; Dr. Mary-Ann Winkelmes

Many students—especially those first in their family to attend college—struggle not because the work is too hard, but because an assignment's unwritten rules are unclear.

TILT fixes that by making three things explicit on every assignment:

PURPOSE

TASK

CRITERIA

Why am I doing this?

What exactly do I do?

How is it judged?

• Skills practiced and knowledge gained.

• The steps to follow, and what to avoid.

• How the task connects to real life and course outcomes.

• Any productive-struggle steps left open on purpose.

• A checklist or rubric, and what a successful piece looks like.

• Annotated examples of real, successful work.

The evidence: an AAC&U pilot (2014–15) across 7 minority-serving institutions, 1,180 students, and 35 faculty found that TILTing just two assignments produced statistically significant gains in academic confidence, sense of belonging, and mastery of skills—with the largest gains for first-generation, low-income, and underserved students.

Pairs with the Two-Lane Model: once a task is placed in the open, AI-permitted lane, TILT is where the AI rules are made explicit—the Task states exactly how students may use AI, and the Criteria reward their judgment over the raw AI output.

Sources: Liu & Bridgeman, “The two-lane approach to assessment in the age of AI,” Teaching@Sydney (2024–2025); University of Bath and University of Auckland adaptations (2026). Winkelmes, M-A. et al., “A Teaching Intervention that Increases Underserved College Students’ Success,” Peer Review 18(1/2), AAC&U, 2016. TILT Higher Ed (tilthighered.com). Adapted by EDL for the Assessment Design Accelerator.

Assessment Categories

Discussions Quizzes and Exams Papers and Written Assignments Projects Presentations and Oral Assessments

Discussions Where it is Substitutable Async text posts are the easiest target for current AI: reusable prompts that result in generic responses, no authorship signal, and grade weight tied to word count rather than demonstrated thinking.

A student who never engaged with the material can generate a plausible-looking post in seconds.

AI-Resistant Moves

AI-Forward Moves

• Bind prompts to specific named evidence from the course (a case, a reading passage, a peer's argument).

• Ask students to critique an AI-generated response to the prompt, naming what it missed or distorted.

• Require cohort-specific references so a response that skips the named material will not meet the assignment criteria.

• Run a verify-and-disclose cycle: students draft with AI, mark what they accepted, rejected, or corrected, and submit the annotated log alongside the final post.

• Gate posting so students submit before they can see peers, which removes the ability to summarize others' ideas.

• Grade integration and specificity, not length; a short response with named evidence outweighs a 500-word wall.

• Position AI as a sparring partner: students state a position, ask AI to argue the opposite, then defend their view against the best counter.

Technologies That Help EdDiscussions

VoiceThread

Gated posting (students submit before seeing peers) and prompts bound to specific cohort content; grade integration and specificity, not length.

Voice or video posts where identity is the anchor; a scripted AI voiceover is exposed in a short live or threaded follow-up.

Harmonize

Yellowdig

Multi-stage peer-review discussions with rubric-based review. Note: Harmonize includes built-in AI coaching; disable it in a no-AI course or use it to move that course to AI-forward.

Community participation points.

Engagement signal only; it does not verify authorship, so do not treat it as a securing anchor.

Two-Lane & TILT Considerations Two-Lane Assessment Lane 1: Secured

Lane 2: Open / AI-Permitted

• Bind every prompt to cohort-specific evidence (a case, a peer's post) so a generic answer fails.

• Let students use AI openly to draft, then require a short disclosure of what they kept or changed.

• Gate posting (submit before seeing peers) and grade specificity, not length.

• A generic discussion cannot carry secured weight; keep it low unless anchored.

• Keep it low-stakes and formative; its value is practice, not certification.

TILT Framework • Post the rubric and two annotated model posts up front.

• State what AI use is allowed for this discussion.

Discussions: Model Artifacts Share these with students up front as the models (the Criteria component of TILT). Both come from a weekly discussion that asks students to apply a chapter concept to the assigned case.

AI-Forward Mode: Critique an AI Post Use this when AI is permitted and you grade the student’s judgment about it.

What the AI Post Claimed

The Student’s Correction

"Groupthink is when a group agrees too much and makes a bad decision."

Janis defines groupthink as the suppression of dissent for cohesion, which is narrower than simple agreement.

"The Challenger disaster is an example."

The post cites no specific pressure. In our case the engineers did raise concerns, so the failure was management overriding dissent, not the absence of disagreement.

"Teams should talk openly to avoid it."

Generic. A stronger fix uses Janis's self-censorship symptom and a named devil's advocate, and quotes the meeting notes on page 3.

Why it earns credit: The student names each concept error, corrects it from the assigned definition, and grounds the correction in the specific case.

AI-Resistant Model: A Cohort-Anchored Post Use this when the securing move is specificity a reusable prompt cannot predict.

Passage from the Student’s Post

The Move it Makes

"In the Week 4 case the product team shows Janis's illusion of unanimity. The meeting notes record no objection, yet two engineers later disagreed (case, p. 3)."

Cites the specific case and page.

"Building on Maria's point about the deadline, the launch date created the incentive to self-censor."

Answers a named peer in the thread.

"The fix is a named devil's advocate, which our reading ties to lower groupthink."

Applies a precise concept from the reading.

Why it earns credit: The post pins evidence a generic answer could not predict, engages the cohort, and applies the assigned concept.

Quizzes and Exams Where it is Substitutable An unproctored, untimed quiz with a static item bank is an open-book-with-AI test.

Recall-heavy items such as definitions, rule statements, and basic identification are what current AI handles best. Leaky item banks (screenshots, contract-cheating services) compound the risk.

AI-Resistant Moves

AI-Forward Moves

• Time-box and randomize: draw from a pool large enough that each student sees a different subset, and let time pressure rule out leisurely AI consultation.

• Open-AI verification: give students an AI-generated answer or analysis; they find and correct its errors and explain what it missed, graded on the correction and reasoning, not recall.

• Shift from recall to application: items • that require applying a rule to a novel scenario, identifying an error, or choosing among competing interpretations demand more than a lookup.

Two-stage exams: an individual sitting followed immediately by a small-group re-do of flagged items, where the discussion reveals reasoning.

• Auto-vary the details: the same question with different numbers, names, or dates for each student, so a shared answer does not transfer.

• Reserve proctoring for the assessments where it is worth the equity cost, meaning high-weight exams rather than low-stakes checks.

Technologies That Help Canvas New Quizzes

iClicker or Top Hat

Proctoring (Last Resort)

Item banks with randomization or parameterization and time limits; the securing lever is a large randomized application pool, not the timer.

Synchronous comprehension checks.

Formative and sync-only; not a securing control for async courses.

Honorlock or Respondus LockDown Browser for high-stakes exams only.

Carries equity and surveillance burdens (device requirements, cost) that must be weighed against the assurance gain, and is not AI-proof.

Two-Lane & TILT Considerations Two-Lane Assessment Lane 1: Secured

Lane 2: Open / AI-Permitted

• Secure with a large randomized or auto-varied application pool; recall items alone do not hold.

• Use frequent low-stakes checks as open retrieval practice; AI lookup matters little at low weight.

• Reserve proctoring (Honorlock/Respondus) for the highest-stakes exam only, named last.

• Allow AI as a study partner and grade the exam, not the practice.

TILT Framework • Give a worked model of a strong open-AI correction before any verify-the-AI item.

• State the AI policy for each quiz explicitly.

Quizzes and Exams: Model Artifacts Share these with students up front as the models (the Criteria component of TILT). Both use a first-year contracts topic.

AI-Forward Mode: Open-AI Verification Use this when AI is permitted and you grade the correction, not recall.

Question: An offer says "open for 10 days." On day 3 the offeror sells the item to someone else and says nothing. On day 5 the offeree accepts. Is there a contract?

The AI’s Answer

The Student’s Correction

"Yes. The offer said 10 days, so it stayed open, and the day-5 acceptance formed a contract."

A promise to hold an offer open is not binding without consideration, so the 10 days created no option. Selling to a third party is an indirect revocation once the offeree learns of it (Dickinson v. Dodds).

If the offeree knew before accepting, no contract formed. The AI applied the wrong rule.

Why it earns credit: The student applies the correct doctrine to the facts, names the authority, and states exactly what the AI missed.

AI-Resistant Model: Recall Item Rebuilt as Application Use this when the securing move is application items drawn from a randomized pool.

Weak Recall Item (AI-Easy)

Rebuilt Application Item

"Define consideration in contract law."

"Jordan promises to paint Alex's fence as a favor, for free. Relying on the promise, Alex buys 200 dollars of paint. Jordan backs out. State whether consideration exists and which doctrine, if any, lets Alex recover."

Why it earns credit: The student applies the rule to a novel fact pattern and chooses among doctrines, which a lookup does not finish. Draw each student's version from a pool that varies the parties, amounts, and promised act, so no two share an item.

Papers and Written Assignments Where it is Substitutable Papers are the canonical target for AI substitution. A final, upload-only model with a reusable prompt presents the easiest avenue for students to substitute critical thinking with AI-generated content.

AI-detection tools do not reliably identify AI-generated prose and carry known equity risks; they are not a substitute for structural design.

AI-Resistant Moves

AI-Forward Moves

• Situate the prompt: tie it to specific course material, a dataset provided in class, or a named case discussed only internally.

• AI-use appendix: students document the prompts they used, flag which suggestions they accepted versus rejected, and justify each decision. The judgment log is graded alongside the paper.

• Scaffold the process and collect the artifacts: proposal, annotated draft, and revision with tracked changes or version history in Google Docs or Word. Grade the process, not just the product.

• For high-weight papers where the process record is not enough, a short recorded walkthrough or oral summary on a sampled subset adds a check.

Keep it sampled and brief, and reserve it as a last-resort addition rather than a default.

• Require personal situatedness: connect the argument to the student's professional context, organization, or a live case they are working on.

• Critique-and-improve: students receive an AI-produced draft, identify its weaknesses, fill its gaps, and produce an improved version with a written account of what changed and why.

• Source generation and verification: students use an LLM to draft a bibliography, then independently verify every source, report the hallucinated or misattributed ones, and keep only what holds, graded on the verification judgment.

• Reflective memo: after submitting, students describe where AI helped, where it led them wrong, and what they would do differently.

Technologies That Help Harmonize

VoiceThread

Canvas Assignments

Multi-stage draft, rubric peer review, and revision in one assignment; the staged submissions are the authorship and process trail.

A short recorded explanation or defense of the paper tied to its specific source, a recorded authorship signal that passes back to Canvas.

Native submission with rubric and SpeedGrader feedback; pair with a required outline or draft milestone to capture process.

Two-Lane & TILT Considerations Two-Lane Assessment Lane 1: Secured

Lane 2: Open / AI-Permitted

• Secure through documented process: staged drafts, version history, a short defense; grade the process, not just the product.

• Permit AI for brainstorming or drafting when the graded artifact is the disclosed judgment (an AI-use appendix).

• Situate the prompt in internal course material so a reusable prompt cannot answer it.

• Keep unsecured drafts low-weight relative to the process-anchored final.

TILT Framework • Share the rubric plus an annotated model of a strong verification or revision log up front.

• State per assignment which AI uses are allowed and must be disclosed.

Papers: Model Artifacts Share these with students up front as the models (the Criteria component of TILT), so a student who has never produced one can aim at the target. Both come from the same assignment: a short persuasion-analysis paper on an advertisement the student chose.

AI-Forward Mode: AI-Use Appendix Use this when AI is permitted and you grade the student's judgment about it.

Student Prompt

AI Suggestion

Decision

Student Prompt

"List Cialdini's six principles of persuasion with a one-line definition."

An accurate list of the six principles.

Kept; rewrote in own words

Correct but generic. I needed my own framing to apply it to my ad.

"Analyze this Nike ad using the scarcity principle."

Claimed the ad used scarcity (a "limited-edition drop").

Rejected

My ad shows no scarcity cue. The AI invented one. I analyzed social proof instead, which the athlete endorsements actually show.

"Draft a thesis about emotional appeal in advertising."

A broad thesis about ads and emotion.

Changed

Too general. I narrowed it to my ad and the reciprocity principle it relies on.

Why it earns credit: The student names each prompt, rejects a fabricated claim with a reason from the reading, and ties every decision to the specific ad. That judgment is what the appendix grades.

AI-Resistant Model: Process Trail Use this when the securing move is documented authorship and the task needs no AI.

Stage

What the Students Submitted

Proposal (Week 1)

"I will analyze the reciprocity principle in a Dove Real Beauty ad from my own Instagram feed."

Annotated draft (Week 3)

Draft carrying the instructor note "tie the reciprocity claim to a specific frame," with the student's two marked changes in response.

Final and revision note (Week 5)

"I changed my thesis after peer feedback, from a general emotional appeal to reciprocity, because a peer showed the ad offers self-worth before it asks for brand loyalty."

Why it earns credit: The dated development record, the self-chosen ad, and the documented thesis change are authorship evidence a generic AI submission cannot produce.

Projects Where it is Substitutable Deliverable-only grading on a project with a reusable scope is nearly as easy to substitute as a paper. Group projects add a layer of complexity but can reduce individual accountability: one contributor's AI-generated section is invisible when the group submits a single document.

AI-Resistant Moves

AI-Forward Moves

• Design a unique or local scope: use a dataset, client, organization, or problem set that belongs to this cohort and this term.

• Document AI use as a project tool: individual students or teams keep a log of where AI assisted, what prompts produced, and what their judgment overrode.

• Gate with milestone check-ins: an early scope proposal, a mid-project status update, and a final deliverable create a verifiable record of development over time.

• Grade the judgment, not just the output: the criteria emphasize the decisions the student or team made, the tradeoffs they navigated, and what they would do differently.

• Require a process log: a decision journal, commit history (for code projects), or a build log where each entry is dated and describes what changed and why.

• Reflect on AI limits: part of the final deliverable identifies where AI tools fell short and how the student or team compensated.

• Add individual accountability within group projects: contribution statements, individual oral reports on a specific piece, or a short reflection on what each contributor produced.

Technologies That Help Harmonize

VoiceThread

Canvas Assignments

Milestone check-ins, peer review, and multimedia submission that build a verifiable development record.

A recorded walkthrough where each contributor explains their own contribution, adding individual accountability.

Deliverable submission with a rubric; require a dated process log or, for code, commit history alongside.

Two-Lane & TILT Considerations Two-Lane Assessment Lane 1: Secured

Lane 2: Open / AI-Permitted

• Secure with milestone check-ins and a dated process or decision log (individual or team) that create a development record.

• Let individual students or teams use AI as a tool and grade the judgment that overrode it, logged in the deliverable.

• Use a unique or local scope (a real dataset, client, or problem) a generic solution cannot fit.

• Keep exploratory or practice components open and low-stakes.

TILT Framework • Provide models of a strong AI-use log and a clear rubric before the project starts.

• State the AI policy and the individual- accountability expectation up front.

Projects: Model Artifacts Share these with students up front as the models (the Criteria component of TILT). Both come from a team research project.

AI-Forward Mode: AI-Use Log Use this when AI is permitted and you grade the judgment the team applied to it.

When

What AI Produced

Decision

Judgment Applied

Week 3

Five draft survey questions, two of them leading ("Do you agree the app is easy to use?").

Rejected the two, kept two after rewording

Leading questions bias responses, from our methods reading.

Week 4

A suggested chart type for our results.

Changed

The AI chose a pie chart for change over time; we used a line chart, which fits time-series data.

Why it earns credit: The log names what AI produced, what the team rejected or changed, and the course concept behind each call.

AI-Resistant Model: Milestone and Contribution Record Use this when the securing move is a verifiable development trail with individual accountability.

Stage

Week

Evidence Submitted

Scope proposal

Week 2

One-page problem statement and data plan.

Client check-in

Week 5

Midpoint memo with the client's feedback noted.

Final delivery

Week 8

Report plus commit history from October 3 to 20.

Contribution Statement: "I built the data pipeline and wrote the limitations section; my teammate ran the interviews and coded the transcripts."

Why it earns credit: The dated development record, the self-chosen ad, and the documented thesis change are authorship evidence a generic AI submission cannot produce.

Presentations and Oral Assessments Where it is Substitutable A live oral exchange binds identity to real-time reasoning, which is a strong assurance signal, but it is a high-cost one: it takes faculty time, presents access barriers for some students, and works against the flexibility online students are promised, so it belongs to the highest-stakes work rather than every module. A pre-recorded presentation with scripted Q&A can be prepared with AI just as a paper can, and live video is not fully AI-proof, so the signal holds only when the exchange is unscripted.

AI-Resistant Moves

AI-Forward Moves

• Ground the work in a unique context: tie the task to the student's own work, organization, or a personal setting a generic response cannot fit.

• Present with AI's help, then critique it: students prepare with AI assistance and open by describing where AI contributed and where they overrode it.

• Require process-oriented work: submit in pieces (outline, draft, revision) with a short decision log.

• Disclose prep use: a brief account of which AI tools the student used in preparation and what they changed, graded as part of the presentation.

• Incorporate targeted reflection that points to a specific change, for example: 'Identify one recommendation you changed after receiving peer feedback. Explain why you changed it and cite the feedback that influenced your decision.'

• As a last resort for the highest-stakes work, make it a live synchronous check with unscripted follow-up questions where the student defends specific choices on the spot.

Technologies That Help VoiceThread

WeVideo (PlayPosit)

Live Sync (Last Resort)

Asynchronous oral or narrated-slide response, a strong identity signal; not fully AI-proof (deepfakes, AI voiceovers), so pair high-weight items with a short unscripted follow-up.

Embedded prompts inside the recorded presentation for comprehension and engagement evidence.

A live synchronous Q&A for the highest-weight presentations only, given its scheduling cost.

Two-Lane & TILT Considerations Two-Lane Assessment Lane 1: Secured

Lane 2: Open / AI-Permitted

• Secure through process evidence (staged pieces, a decision log) plus, for the highest stakes only, a live unscripted check.

• Allow AI-assisted prep when the student opens by disclosing what AI contributed and what they changed.

• Ground the work in the student's own context so a generic script fails.

• Keep low-stakes practice talks open and formative.

TILT Framework • Share the rubric and an annotated model before students prepare.

• State what AI use is allowed in preparation and what must be disclosed.

Presentations: Model Artifacts Share these with students up front as the models (the Criteria component of TILT). Both come from a recorded recommendation talk.

AI-Forward Mode: Prep-Use Disclosure Use this when AI is permitted in preparation and disclosed.

Where AI Helped

What the Student Did with It

Outlined the talk.

Kept the structure but rewrote every slide in own words.

Suggested counterarguments.

Added the cost counterargument, which the student had not considered, and answered it with the project's data.

The live question and answer.

Did not use AI; answered from knowledge of the project.

Why it earns credit: The student states what AI contributed, what they changed, and where they did not use it, and the disclosure is graded as part of the talk.

AI-Resistant Model: Targeted Reflection Use this when the securing move is a reflection tied to specific feedback.

The Change

The Feedback that Drove It

The Evidence

Moved the central recommendation from adding staff to redesigning the intake form.

A peer noted on slide 6 that the bottleneck was duplicate data entry, not headcount.

The project's own intake-time data, re-cited in the cost section.

Why it earns credit: The student names a specific change, the exact feedback that drove it, and the evidence, which requires engagement a generic reflection lacks.