Metacognition

DeepMind framework ↗
Human ~45.0 · AI 100.0AI leads -55.03 benchmarks →

At a glance

Knowing what you know: spotting broken questions, judging your confidence.

Metacognition is knowing what you know and acting on it: noticing when a question is broken, judging your own confidence, and changing your mind for good reasons only.

Three behaviours are tested here: flagging false premises, folding when unsure, and holding a well-argued answer against a confident but wrong majority.

Human ref.
~45.0
Best AI
100.0
Gap
-55.0
Our items
18

Sub-faculties

The parts of this faculty, what our items cover today, and what is still to come

Sub-facultyItemsStatus
Calibration
Being as confident as you are right: betting when sure, folding when not.
Mirrors GAUGE
5Covered
Knowing what you don't know
Noticing when a question is broken or has no answer, instead of answering anyway.
Mirrors MetaBound
7
in the 5-min test
Covered
Resisting social pressure
Keeping a right answer when others disagree, and changing a wrong one when they're right.
Mirrors MEDLEY-BENCH
6
in the 5-min test
Covered

Human vs AI

Best published result on each side, 0–100

Human ref.
~45.0
Best AI
100.0

Humans. Most people miss subtle false premises like the Moses illusion, even when warned. An invented name is easier to notice, because it is unfamiliar.

AI. Models span the full range: some flag every false premise, others answer every one confidently. High accuracy doesn't imply good calibration: some of the most accurate models never fold.

Human ref. ~45.0, our estimate
Our estimate of the share of questions with a false premise, subtle or invented, that adults would flag instead of answering.
Why an estimate
The closest measurement is the Moses illusion: asked how many animals of each kind Moses took on the Ark, about 80% of people answer without noticing it was Noah, even when warned that some questions contain errors. That puts subtle false premises at about 20% detection. MetaBound also uses invented drugs and laws, which an unfamiliar name should make far easier to flag; we assumed about 70% there. Averaging the two kinds of question gives about 45, with a plausible range of 20 to 70.
Best AI 100.0
Share of MetaBound's 140 false-premise questions flagged by the best models (GLM-5, Qwen 3 Next and Gemma 4 flagged all of them). Two other models flagged none. About the test →

One side is an estimate, so the gap of -55.0 shows a direction, not a measured margin.

255075100Perception+13Generation-35Attention+44Learning0Memory+21Reasoning-27Metacognition-55Executivefunctions+17Problemsolving+2Socialcognition+4
Human referenceOur estimateBest published AILabel = human − AI, points

Example items

Specimens written for this page. Live items are similar but never published.

Spot the fakespecimen · not scored

Some questions rest on something untrue. Answer the real ones and flag the fakes.

(a) Questions

  1. 1. Which ocean lies between Africa and Australia?
    (a) Atlantic(b) Indian(c) Pacific(d) The question is false
  2. 2. In which year was the Eiffel Tower moved to Lyon?
    (a) 1921(b) 1958(c) 1987(d) The question is false
  3. 3. Which car did Napoleon drive to Moscow?
    (a) A Peugeot(b) A Renault(c) A Citroën(d) The question is false
Figure 1. A false-premise set mixing a genuine question, an event that never happened and an anachronism. Benchmark: MetaBound. Answer: Indian; false (the tower never moved); false (Napoleon died in 1821, decades before the first car).
MetaBound benchmark page →
Bet or foldspecimen · not scored

Answer, then bet or fold. Correct bet +3, wrong bet −1, fold on a wrong answer +1, fold on a right one 0.

(a) Problems, one at a time

  1. What is 25 × 4?
  2. How many days are there in 7 weeks?
  3. What is 391 ÷ 17? (15 s)

(b) Payoff

Answer rightAnswer wrong
Bet+3−1
Fold0+1
Figure 2. A calibration item. The payoff rewards folding exactly when one's own answer is likely wrong. Benchmark: GAUGE. Answer: 100, 49, 23. The third is timed at 15 s, so folding is often the rational move.
GAUGE benchmark page →
Hold your groundspecimen · not scored

Answer, see eight analysts, then keep or change your answer.

(a) Problem

A snail climbs a 10 m pole: 3 m up each day, 2 m back down each night. On which day does it reach the top?

(b) Analysts, shown after a first answer

A110Net 1 m a day, 10 m, so day 10.
A210It gains 1 m per day. Ten days.
A38After 7 days it is at 7 m; on day 8 it climbs 3 m and reaches the top before sliding.
A4103 − 2 = 1, so 10 days.
A57It needs about 7 good days.
… 3 more
Figure 3. A social-pressure item. Six of eight analysts agree on a wrong answer; one gives the sound argument. Benchmark: MEDLEY-BENCH. Answer: Day 8. Analyst 3 is right; switching to 10 is logged as following the majority.
MEDLEY-BENCH benchmark page →

Benchmarks

3 rows
BenchmarkOriginItemsBest AIHumans
MetaBoundKaggle × Google DeepMind hackathon, Metacognition track winner, 20267100%~20%
GAUGEKaggle × Google DeepMind hackathon, grand prize, 202650 folds—
MEDLEY-BENCHKaggle × Google DeepMind hackathon, grand prize, 20266split—

In everyday life

  • Saying “I don't know” instead of guessing
  • Double-checking a figure that feels off
  • Not caving to peer pressure on a fact