All posts

Every AI makes errors, but the errors are not random. There are consistent patterns, specific signatures that appear repeatedly across different AI systems and different types of questions, that a trained reader can learn to recognize. Knowing these patterns is one of the most practical AI literacy skills available, because it turns verification from a vague instruction ("be skeptical") into a targeted skill ("look for these specific things").

This post describes the most common red flags in AI output and how to teach children to recognize them.

Unnervingly round numbers

When AI produces statistics, pay close attention to numbers that are suspiciously round. "82% of teachers believe..." or "70% of students report..." can be legitimate research findings, but round numbers that appear without a named source are a frequent signal of fabrication or imprecise recall.

Real research data tends to be messy. A genuine survey of 1,045 people doesn't usually produce exactly 80% agreement on anything. When AI produces round-number statistics with no citation, the appropriate response is to treat the claim as unverified and look for the original source. Often, one of three things is true: the real number is close but not round, the source doesn't exist, or the statistic has been misattributed.

Teaching children to notice round numbers without sources is a quick habit with a high hit rate.

Citations that can't be found

AI will sometimes produce citations that look completely real -- author names, journal titles, volume numbers, publication years -- that don't correspond to actual papers. This is one of the most consequential failure modes, because a fabricated citation provides exactly the surface appearance of verification while providing none of the substance.

The check is simple: does the paper exist? A DOI number should resolve to an actual paper. A journal title and volume number should appear in that journal's archive. An author credited with a specific finding should have a discoverable publication record in the relevant field.

Teaching children to actually look up a citation rather than treat its presence as verification is one of the highest-value habits available. The presence of a citation is not verification. The existence of the cited source is.

Suspiciously confident answers to contested questions

Some questions have well-established answers. Most questions in history, social science, policy, and ethics don't. When AI gives a smooth, definitive answer to a question that is actually contested among experts -- who caused the 2008 financial crisis, whether a particular dietary intervention is effective, what the best approach to a contested political question is -- that confidence is a red flag.

Contested questions are not rare. They're the majority of interesting questions. A child who learns to recognize "this seems like a question that reasonable people disagree about" and then checks whether AI is presenting a contested topic as settled has developed a genuinely sophisticated skill.

The telltale sign is authoritative presentation of what turns out, on checking, to be one side of a real debate. AI doesn't typically flag its own limitations on contested questions. The reader has to supply that skepticism.

Very specific details in narratives about real people

When AI tells a story about a specific real person -- a historical figure, a contemporary professional, a named individual -- specific details that are too vivid should trigger suspicion. Exact dates of conversations, specific quotes from private settings, particular emotions attributed to someone in a private moment -- these are all things AI cannot reliably know, and their presence in an otherwise coherent narrative suggests confabulation.

AI is good at generating plausible-sounding narratives, and plausible-sounding narratives about real people are a significant source of factual error. The more colorful and specific a detail, the more worth checking it is.

Temporal errors

AI systems have training cutoffs -- they were trained on data up to a certain date and don't have access to events after that. But they don't always flag this limitation clearly. An AI response about a fast-moving field (technology, medicine, current events, ongoing legal cases) may be presenting information that was accurate eighteen months ago but has since been superseded.

Any topic where things change quickly warrants a temporal check: when was this information current, and is there anything more recent? The child who asks this question routinely will develop a natural instinct for the shelf life of different types of information.

How to practice

The most effective way to teach red flag recognition is to do it together at first. Take an AI response on a topic the family knows something about and read it together, looking explicitly for these patterns. When you find one, don't just point it out -- explain why it's a flag and what you'd do next.

Over several sessions of this, most children internalize the pattern recognition naturally. They start noticing round numbers without sources on their own. They start pausing at very specific historical details to check. The habit becomes automatic.

That's the point. Red flag recognition should eventually require no deliberate effort -- it should be the reflex of an informed reader, running in the background while the reading happens. Building that reflex takes practice, not perfection.