Limitations of Generative AI: The Awkward Truth of What’s Wrong
Generative AI’s primary limitations include producing factually incorrect information (hallucinations), inheriting biases from its training data, and posing…

Generative AI creates Multiple Choice Questions (MCQs) by analyzing source material with a large language model to produce a question, a correct answer, and several plausible incorrect options (distractors). This automates content creation for educators but requires careful prompting and human review to ensure quality and accuracy.
So you want to test your knowledge with some generative AI questions and answers. Fair enough. But let’s be honest, the more interesting question isn’t what the answers are, but how an AI can generate a quiz in the first place. That’s where the real power lies.
AI-written content isn’t the problem. Briefless AI-written content is. Anyone can type “write me a quiz about generative AI” into a chatbot and get a list of questions. But creating a good quiz—one that’s accurate, fair, and actually tests understanding—is a different game entirely. It requires a process.
This guide is that process. We’ll skip the generic, low-effort list of 30 questions you could generate yourself in two minutes. Instead, we’ll show you how to become the quiz master. You’ll learn how to use AI to create high-quality multiple choice questions from your own material, how to refine them, and what to watch out for.
Using a Large Language Model (LLM) like ChatGPT or Claude to generate MCQs feels like magic. You paste in a chapter from a textbook, write a simple prompt, and get a stack of questions back. For educators, marketers creating lead magnets, or corporate trainers, this seems like a dream.
And it can be. But it’s not “set it and forget it.”
Think of the AI as a hyper-caffeinated teaching assistant who has read the source material once, very, very fast. It’s brilliant at spotting key terms and facts, but it lacks the judgment of an experienced teacher. It can’t truly grasp learning outcomes, student context, or nuance without explicit instructions.
The trick isn’t just using the tool; it’s building a resilient system around it. A good process turns the AI from an unreliable intern into a powerful assistant for any educator.
An AI-generated MCQ is a multiple-choice question created by a generative AI model. The process involves feeding the AI a piece of source content (like an article, a video transcript, or a PDF). The model then analyzes the text to create three key components:
Here’s a simple example. Imagine we gave an AI an article about the basics of Large Language Models. It might produce something like this:
Question: What is the primary architecture used in modern Large Language Models like GPT-4?
a) Variational Autoencoder (VAE)
b) Generative Adversarial Network (GAN)
c) Transformer
d) Recurrent Neural Network (RNN)
The AI identified a key fact (Transformer architecture) and generated plausible-sounding but incorrect distractors from the broader field of machine learning. When it works, it’s incredibly efficient.
Resist the urge to automate everything at once. The smart path is to prove one workflow first. Start by creating a small batch of 5-10 questions using this simple but effective framework.
This is our simple, three-part loop for getting quality results.
The model needs context. Don’t just ask it to write a quiz on a topic; give it the specific document you want the quiz to be based on. This connects the AI directly to the source of truth, dramatically reducing the risk of it pulling in random (and potentially wrong) information from its general training data.
You can usually just copy and paste the text directly into the chat window. For longer documents, use models that have a large context window, like Claude, or upload the file directly if the tool allows it.
Your initial prompt doesn’t need to be complicated. Start with a clear, direct request. The real work is in the refinement later.
Here’s a solid starting prompt you can use:
`Act as an expert instructional designer. I will provide you with a text below. Your task is to generate [number] multiple choice questions based on this text.
For each question:
- Ensure it is clear and unambiguous.
- Provide one correct answer.
- Provide three plausible but incorrect distractors that target common misunderstandings.
- Clearly indicate the correct answer.
Here is the text:
[Paste your source text here]`
This prompt sets the role, specifies the quantity, and defines the structure of the output. It’s a great starting point for any subject.
This is the most important step. The AI’s first draft is just that—a draft. Your job is to act as the editor.
Look for:
Use your chat history to refine the output. You can give feedback like:
There are dozens of AI tools out there, but most people only need one of the big three general-purpose models. Don’t overthink it. Most of these tools are fine, a few are great, and some are inexplicably expensive for what they do. Start here.
| AI Tool | What It Is | Who It’s For | Pros & Cons | The Standout Feature |
|---|---|---|---|---|
| ChatGPT (GPT-4) | The most well-known and versatile large language model from OpenAI. The paid version (GPT-4) is significantly more capable than the free one. | Everyone. It’s the best all-rounder for educators, marketers, and anyone getting started. | Pros: High-quality output, great reasoning, can handle complex instructions. Con: Free version (GPT-3.5) is noticeably less accurate. |
Overall Reliability: GPT-4 consistently produces the most logical and well-phrased questions and distractors on the first try. It’s the main character of AI tools for a reason. |
| Anthropic’s Claude 3 | A powerful competitor to ChatGPT with a strong focus on safety and a massive context window for analyzing long documents. | Educators with long source texts. If you need to feed it a 100-page PDF, Claude is your best bet. | Pros: Huge context window (up to 200k tokens), excellent at summarizing and pulling facts from dense documents. Con: Can sometimes be a bit “wordy” or overly cautious in its responses. |
Document Analysis: Nothing else handles massive documents with the same ease. Perfect for generating a quiz from an entire book chapter or research paper. |
| Google Gemini | Google’s flagship generative AI model, tightly integrated with its ecosystem. It’s rapidly improving and a strong contender. | Users in the Google ecosystem or those looking for a free, powerful alternative. | Pros: Fast, often creative with its output, and the advanced version is competitive with GPT-4. Con: Can sometimes feel less polished or consistent than its main rivals. |
Real-Time Information: Its integration with Google Search means it can sometimes pull in very current information, though this can be a double-edged sword for quiz creation. |
Our Verdict: Start with the paid version of ChatGPT (using the GPT-4 model). It offers the best balance of power, reliability, and ease of use for generating high-quality generative AI MCQs. If you’re working with very long documents, give Claude a try.
“Prompt engineering” is a fancy term for giving the AI better instructions. It’s a skill, not a career. Once you’ve mastered the basic prompt, you can add layers to get much more sophisticated results.
Most basic AI questions test simple recall (Level 1 of Bloom’s Taxonomy). To create questions that test deeper understanding, you need to ask for them specifically.
Bloom’s Taxonomy is just a framework for categorizing levels of thinking, from simple memorization to complex creation. Here’s how to prompt for different levels:
Example Prompt Addition:
Generate 5 questions. Two should test for simple recall of definitions. Three should test for "Application" by asking the user to identify the best example of a concept from a scenario.
You can also request different question formats to vary your assessment and prevent students from just pattern-matching.
AI models are trained on vast amounts of internet text, which is full of societal biases. If you’re not careful, these biases can seep into your quiz questions. You can actively prompt the AI to mitigate this.
Example Prompt Addition:
When generating questions, review them for any potential cultural, gender, or racial bias. Ensure names, scenarios, and examples are diverse and inclusive. Avoid stereotypes.
You ensure quality with a systematic, human-led review. The AI is a tool, not the final decision-maker. A tool is not a strategy. Use this checklist as your quality assurance strategy before any AI-generated question goes live.
Using AI for assessment is powerful, but it comes with significant ethical responsibilities. It’s not just about stopping students from cheating; it’s about ensuring the assessment itself is fair, private, and unbiased.
The obvious concern is that students might use AI to answer questions. But a deeper issue is fairness. If an AI generates a poorly worded or factually incorrect question, it can unfairly penalize a student who actually knows the material. The responsibility for a fair assessment still rests with the educator, not the algorithm. This is why the human verification step is non-negotiable.
When you paste your source material or student data into a public AI tool, where does it go? Be mindful of data privacy, especially with sensitive or proprietary information. Check the terms of service of any AI tool you use.
Furthermore, these models reflect the biases present in their training data. If you’re generating questions about historical figures, social issues, or even technical roles, the AI may inadvertently produce biased outputs that reinforce stereotypes. Actively prompting for inclusivity and carefully reviewing the output is crucial to avoid perpetuating harmful biases in your learning content.
Getting your shiny new questions from a chatbot into your Learning Management System (LMS) like Canvas, Moodle, or Blackboard can feel clunky. Here’s how to streamline it.
The key is to ask the AI to format its output in a way your LMS can easily import. Most systems support common formats like CSV (Comma-Separated Values) or QTI (Question and Test Interoperability).
Example Prompt for LMS Formatting:
`…[your main prompt here]…
Format the final output as a CSV file with the following columns: “questiontext”, “optiona”, “optionb”, “optionc”, “optiond”, “correctanswer”. For the “correct_answer” column, please list the letter of the correct option (e.g., “C”).`
The AI will generate the text in a clean, table-like structure that you can copy and paste into a text file, save as a .csv, and import directly into your LMS quiz builder. This saves a massive amount of manual data entry.
Here’s where you can really level up. The true value of AI in assessment isn’t just generating the questions—it’s generating the feedback. This moves the AI from a simple content generator to a genuine teaching partner.
After you’ve generated your MCQs, you can use a follow-up prompt to create explanations for each answer.
Example Prompt for Feedback:
`For the multiple choice question below, please provide a brief explanation for why the correct answer is right and why each of the distractors is wrong. Tie your explanation directly to the source text provided earlier.
[Paste the AI-generated question and answers here]`
This gives students immediate, targeted feedback, helping them understand why they made a mistake instead of just telling them they were wrong. It’s one of the most powerful and underutilized applications of generative AI for educators.
Even with a great process, things can go wrong. Here are the most common pitfalls:
Can AI generate other types of questions besides MCQs?
Yes. You can prompt generative AI to create a wide variety of assessment items, including true/false questions, fill-in-the-blanks, short-answer questions, and even complex scenario-based essay prompts. You just need to clearly define the desired format in your prompt.
How much time does using AI to create MCQs actually save?
It can save a significant amount of time, especially in the initial drafting phase, which is often the most time-consuming part for item writers. A human expert might spend 15-30 minutes writing and refining a single, high-quality MCQ. An AI can generate a solid first draft of 10 questions in under a minute. The real savings come from automating the “blank page” problem, allowing you to focus your time on refinement and quality control.
Is it possible for AI to create a full quiz with images or diagrams?
Yes, but it’s more complex. Multimodal models like GPT-4 and Gemini can analyze images and diagrams. You could, for example, upload a diagram of the water cycle and ask the AI to generate questions about it. However, generating novel images (using models like DALL-E) to go along with questions is a separate step and requires more advanced workflows. For now, the most reliable method is generating text-based questions about existing visual material.
Generative AI’s primary limitations include producing factually incorrect information (hallucinations), inheriting biases from its training data, and posing…
Generative AI for business refers to artificial intelligence systems that create new, original content—like text, images, and code—from…
Generative AI examples include tools that create original content across various formats. Common applications are text generation with…