BULLETPROOF QUIZZES

Lewis Carr crusades for truth, justice and the LMS way, in the never-ending battle against cheaters, glitches and creative workarounds in your quizzes.

As an educator or training manager, you know that online quizzes and assessments are essential to your course delivery. However, building effective quizzes that deter cheating and produce reliable results can be a big undertaking. From technical glitches to creative user workarounds (also known as cheating), many potential pitfalls can undermine the integrity of your assessments.

In this article, we’ll explore strategies and best practices for creating ‘bulletproof’ online quizzes that deliver accurate, trustworthy results - no matter your obstacles. By the end, you’ll have a toolbox of techniques to build rock-solid assessments that uphold academic honesty and are as bulletproof as Batfink’s wings (Google it if you’re not a child of the 80s).

Why Bulletproof Quizzes Matter

There’s no doubt that online quizzes offer benefits not only to you as the instructor but also to your learners. They provide instant feedback, streamline grading and create opportunities for more frequent, low-stakes evaluations. Sounds like a win-win, right?

Well, not quite. The very features that make online quizzes so convenient and valuable also introduce risks and vulnerabilities. Without the physical supervision of an in-person exam, learners have more opportunities to cheat. That’s why every school had a Mrs Badcrumble, scarily walking up and down the exam hall, watching your every move as you flicked through your GCSE science paper. If you even tried to look at those notes inked onto your arm, she would see you.

Online learners will try and cheat by using an array of unauthorised resources, either collaborating with peers by asking Dave from the warehouse to help or employing AIpowered tools to generate their answers.

Cheating on online assessments is a growing problem that undermines the validity of your course data, skews your performance metrics, and ultimately cheats users out of the required learning. A recent survey in the U.S found that over 40% of college students admit to cheating on online exams. If college students do this, you can bet that workplace employees are doing it, too. Heck, you’ve probably even done it yourself (remember that drag-and-drop fire extinguisher training you did? Which colour was the foam one again?).

Beyond the ethical concerns, insecure quizzes can also expose you and your organisation to serious legal and compliance risks. Colleges and universities have strict academic honesty policies, and failure to uphold them can lead to disciplinary measures, damaged reputations, and sometimes, legal action. In the workplace, ineffective compliance training has similar, and sometimes even more serious, repercussions. How would you feel if your dentist didn’t actually pass their exams and they cheated on a quiz? How confident would you be about that tooth extraction now? (Yes, I know dentists don’t take Moodle quizzes, but you get my drift).

That’s why building bulletproof online quizzes in your organisation should be a top priority. By proactively implementing robust security measures and smart design principles, you can create assessments that are more resistant to cheating, provide performance data you can trust, and give learners the feedback they need to truly learn. OK, they might not be completely bulletproof, unlike Batfink. But stopping some bullets is better than none at all.

Key Principles of Bulletproof Quiz Design

At the heart of building bulletproof online quizzes are a few key design principles that you should keep in mind:

Randomisation

Randomise the order of questions, answers, and even question types to make it harder for learners to coordinate or share answers. This makes creating a ‘cheat sheet’ more difficult. I’ve seen too many American high school movies to know that someone somehow manages to get hold of the answers to next week’s history test.

Start by randomising the order of questions within each quiz. This ensures learners see a unique sequence of questions rather than a predictable order that could be shared among other learners (Dave from the Warehouse might still be able to answer them for you, though, but more on that later). Many LMS platforms offer built-in randomisation features to make this easy to implement.

Next, randomise the order of answer choices for multiple-choice and true/false questions. This prevents learners from simply memorising the position of the correct answers. Some LMS tools even allow you to randomise the order of answer choices for each individual learner.

Finally, we can randomise the quiz by drawing from a larger pool of questions. Instead of using the same set of questions for each learner, your LMS can select a unique subset from your question bank. This ensures that no two learners see the same quiz, even if they take it simultaneously. Take that, Dave from the warehouse!

Time Limits

Set strict time limits on quiz completion to discourage using Google, phoning a friend or asking ChatGPT. Adding time limits to quizzes or questions means that Googling for answers, copying and pasting into ChatGPT, or running down to the warehouse wastes valuable time. When setting time limits, aim to strike a balance between challenging learners and avoiding undue stress. A good rule of thumb is to allocate 1-2 minutes per multiple-choice or true/false question and 5-10 minutes per shortanswer or essay question. You can adjust these estimates based on your questions’ difficulty level and length.

Some LMS platforms even allow you to set different time limits for individual questions or sections of the quiz. This can be particularly effective for larger assessments, as you can allot more time for complex questions that require deeper thought.

Beyond just setting the clock, you can leverage other time-based features to enhance the security of your online quizzes. For example, many LMS tools can automatically submit a quiz when the time expires, preventing learners from continuing to work on the assessment. You can also disable the ability to pause, save, and resume the quiz, forcing learners to complete it in a single session. “When the bell sounds, put down your pens, please”.

Question Variety

Use a mix of question formats (not just multiple-guess); short answer questions require the learner to demonstrate understanding. Using drag-and-drop, sorting exercises, and question types that are difficult to paste into other tools makes cheating more challenging (but not impossible).

Each format tests different cognitive skills and knowledge, forcing learners to engage more deeply with the course material. Furthermore, the diversity of question types makes it nearly impossible for learners to anticipate the structure of the quiz and prepare generic, onesize-fits-all cheating strategies like those pesky high-schoolers did in the movies.

You can also get creative with the content and format of your quiz questions. Rather than relying solely on text-based prompts, consider incorporating multimedia elements like images, videos, or audio clips. This makes the assessment more engaging and introduces new variables that can trip up would-be cheaters.

For example, you could ask students to analyse a chart or diagram, identify key features in an image, or transcribe information from an audio recording or video. These questions are much more challenging to Google and ask AI, as they require a genuine understanding of the course concepts. OK, Dave from the warehouse may still be able to help Steve, but at least he’d have to sit next to him and watch it together.

Adaptive Difficulty

Leveraging adaptive learning algorithms to dynamically adjust question difficulty based on learner performance makes it harder to game the system. Big words, I know, and it sounds technical as hell, but some LMS platforms do this.

You could base this on previous quiz scores, baseline, or aptitude tests. Or go old school and when you speak to the learner, quiz them in person and then adjust their pathway based on their in-person answers.

Adaptive learning algorithms on LMS platforms use sophisticated analytics to dynamically adjust the difficulty of quiz questions based on each learner’s performance. As learners progress through the assessment, the algorithm continuously evaluates their responses and adjusts the next question accordingly. There’s some clever witchcraft going on here.

Robust Lockdown

You can implement robust security features like browser lockdown, IP restrictions, and webcam monitoring to limit opportunities for cheating. Yes, this sounds like the stuff of George Orwell’s 1984, but proctoring software is pretty mainstream these days. IP lockdown won’t stop a user from “What’s Apping” Dave, or even having their gran sit next to them for the test, but it would prevent someone in another location from answering for them. Web cam monitoring, however, is pretty bulletproof. It’s important to strike a balance between security and learner privacy when implementing these lockdown features. Be transparent about your monitoring practices, clearly communicate expectations, and accommodate learners with accessibility needs or technical limitations. But, when all is said and done, webcam monitoring would mean someone would have to go all Norman Bates and dress up as their mum in order to take the test for them (or deep fake it with AI).

Confidence-Based Questions

There’s nothing like being asked a question and then, after you answer, rating your confidence. It’s what Chris Tarrant used to do on Millionaire. ‘Are you sure? Is this your final answer?’. It makes you doubt yourself. With confidence marking, you can score the answer differently depending on whether the answer was correct or not and offset this against their confidence rating. This reduces the chances of multiple guesses; you can see from the data that someone is cheating or guessing. You can penalise those who guess.

Continuous Monitoring

This next one is labour-intensive for you, but you can monitor learner activity, identify suspicious patterns, and investigate potential academic integrity or workplace breaches. Start by closely monitoring learner activity and performance during the quiz. Many LMS platforms offer detailed analytics and reporting features that can help you identify suspicious patterns, such as unexpectedly high scores, rapid changes in performance, Identical answers among learners and unusual login or submission times. It is old school and reminds me of back in the day when teachers used to know if you’d copied Helen’s homework.

Final thoughts

Remember, building bulletproof assessments isn’t just about catching cheaters - it’s about upholding integrity, promoting authentic learning, and giving your learners the formative feedback they need on their quizzes. Yes, you will catch some cheaters along the way, but with the right strategies and tools, you can transform your standard online quizzes into ironclad, trustworthy evaluations that are “like a shield of steel”.