All posts

When a child says "I'm pretty sure about this," what does that mean? In most households, it means roughly what it sounds like -- a vague expression of moderate confidence. But in a world where AI outputs uniformly confident text regardless of underlying accuracy, and where children will spend their professional lives making decisions under uncertainty, "pretty sure" is not a calibrated position. It's a feeling.

Calibration means matching how certain you say you are to how certain you actually have reason to be. A well-calibrated person who says they're 80% confident about something is right about 80% of the time. A poorly calibrated person who says they're 80% confident might be right 50% of the time, or 95% of the time; the expressed confidence doesn't correspond meaningfully to their actual accuracy.

Research in decision science consistently shows that most people are systematically overconfident -- they say they're more sure than the evidence warrants. AI literacy practice is, among other things, practice at catching and correcting overconfidence: in AI's outputs, and in your own.

The basic exercise

The confidence calibration exercise can be run in ten minutes with no materials. It works best as a periodic check rather than a one-time event.

Present a set of ten factual questions. Choose a mix of easy ones and harder ones, across different domains. After answering each question, ask the child to rate their confidence: How sure are you that you're right? Have them write down a percentage or a level (very sure, somewhat sure, not sure, guessing).

Then check the answers. Now count: of the questions they said they were very sure about, what percentage did they actually get right? Of the ones where they said they were somewhat sure, what percentage were correct?

If a child says "very sure" on six questions and gets five right, that's pretty well-calibrated at the top level. If they say "very sure" on six questions and get only three right, they're systematically overconfident at that level -- and that's a useful thing to know.

For younger children, this exercise works with just three levels: sure, kind of sure, not sure. The precision isn't the point; the habit of asking "how sure am I, really?" before committing to a confidence level is.

Connecting to AI

Once a child has done this exercise a few times with their own knowledge, apply it to AI output. Read an AI response together. For each distinct factual claim, ask: how confident does AI seem about this? Now ask: based on what you know, how confident should it actually be?

The exercise almost always reveals the same pattern: AI expresses roughly equal confidence across claims that range from well-established to completely fabricated. The tool doesn't modulate. The reader has to supply the calibration that AI doesn't provide.

This is the moment to name what's happening: AI's confident tone is a feature of its design, not a signal of its accuracy. Reading it critically requires bringing your own calibration to it rather than importing AI's calibration as your own.

The transferable skill

Calibration practice is valuable far beyond AI literacy. Decision-making research shows that well-calibrated people make better decisions in uncertain environments -- not because they know more, but because they're more accurate about the boundary between what they know and what they don't. That boundary, clearly perceived, is what makes uncertainty navigable.

A teenager who has practiced confidence calibration for a few years knows when to say "I'm not sure about this and should check before acting on it" versus "I'm confident enough to proceed." That judgment call is one of the most consequential things any person makes repeatedly throughout their life. Getting it right, more often than not, is a learnable skill.

The AI context makes it more urgent but not different in kind. The same person who modulates their confidence appropriately around AI output will modulate it around medical advice, financial claims, and political assertions. The habit generalizes. Starting it with something as concrete and checkable as AI output is an unusually good entry point.

A note on perfectionism

Some children resist the confidence calibration exercise because they don't want to be wrong. If being wrong feels catastrophic, the exercise surfaces that fear rather than building a skill.

The reframe that works: calibration is about being accurate about your uncertainty, not about being right. A person who said "I'm 50% sure" and turned out to be wrong wasn't wrong -- they were appropriately calibrated. A person who said "I'm 95% sure" and turned out to be right but happened to have been lucky -- that's actually a calibration failure worth noticing.

The goal is not to know everything. The goal is to know, accurately, what you know and what you don't. That standard is achievable, intellectually honest, and more useful than perfect recall.