Home › Lessons › How AI gets things wrong
How AI gets things wrong
Not one failure but five, and they fail differently. It invents details, answers from last year, cannot check where anything came from, answers questions it should decline, and agrees the moment you push back.
Is it making this up?
Would you trust that answer?
That is the exact shape of a real citation, which is why it convinces. An AI can produce the shape without the substance: plausible name, plausible institution, plausible year, and no source behind any of it.
Invented references are the most common way this goes wrong in schoolwork, and they are hard to spot precisely because they look correct.
Is it up to date?
Would you trust that answer?
Hours change, holidays happen, and a pool closes for maintenance without telling an AI. The confident “yes” is doing work the model cannot back.
The check takes ten seconds on the venue’s own page. The habit is the point, not this one answer.
Can it tell what is fake?
Would you trust that answer?
Nothing in that answer involved checking the school, the exam board, or who posted first. The AI evaluated an image and returned a verdict about reality.
The question that closes it: who published this originally, and can I see it there? A screenshot is a copy, and a copy carries no proof of where it came from.
Should you even be asking it?
Would you trust that answer?
Interactions depend on dose, timing, other medication and the person. Every one of those is unknown to the model, and that sentence quietly admits it.
A pharmacist answers this in two minutes, for free, and is accountable for the answer. That is the whole difference.
Does it just agree with you?
Would you trust that answer?
1215 was right. It was given up the moment it was questioned, because the model is shaped to be agreeable and confident, and those two pull in the same direction here.
This is the failure that matters most in schoolwork: a student who pushes back on a right answer gets rewarded with a wrong one, delivered just as confidently. Why it happens is a lesson of its own: the Yes-Person Trap.
Confident doesn’t mean correct.
Don’t ask only “What does AI say?” Ask “How does AI know?”
What this lesson taught
Five ways an AI answer goes wrong, and they are not the same failure wearing different hats.
- It invents detailsA detailed answer is not a checked answer.
- It answers from what it learned, not from todayAI answers from what it learned, not from today.
- It cannot check where anything came fromSeeing something is not verifying it.
- Some questions should not go to it at allSome questions are too important to answer with a guess.
- It agrees when you push backAgreement is not evidence. An AI that changes its answer under pressure never knew.
Was this useful?
Your child meets all five of these
You have just worked through the ways an AI answer goes wrong. A nine-year-old doing homework meets exactly the same five — without having been told any of it. FutureMinds teaches them by playing, not by warning.
Join the conversation
Comments are read by a person before they appear — usually within a day or two. Nothing is published automatically.