Cognitive reflection test
View original ↗At a glance
- Tests
- Quantitative reasoning (Reasoning): checking a quick numerical answer before giving it, because the first number that comes to mind is often wrong.
- How
- Short word problems with a tempting wrong answer: a price split, a rate, something that doubles every day. Getting them right means stopping the intuitive answer and working it out.
- Best AI
- 96%
- Humans
- 38%
- Our items
- 6
- Reasoning gap, overall
- -26.7
Example item
A specimen written for this page. Live items are similar but never shown, so posted answers can't spoil them.
Read the problem, then type the number.
(a) Stimulus
(b) Question
How many cents does the ball cost?
Human vs AI
What the benchmark reports, and where the faculty stands overall
Best published AI
96%
GPT-4 on cognitive reflection tasks (people: 38%)
Hagendorff, Fabi & Kosinski, Nature Computational Science, 2023 ↗
Humans
38%
455 people (Hagendorff et al., 2023)
Reasoning overall · faculty reference, 0–100
Measured on other tests, not on Cognitive reflection test.
Human: GPQA Diamond, PhD experts. AI: GPQA Diamond, Gemini 4 Argon (Epoch AI).
How we mirror it
Six new problems built the same way as the classic ones, which are too widely known to use as they are. Each has one right number and one tempting wrong one.
Typing the tempting answer is logged as a lure, so a slip can be told apart from falling for the trap.
Items are original, versioned, and assigned at random, so a posted answer spoils only one variant.
What we record