CUTE
View original ↗At a glance
- Tests
- Reading text precisely (Perception): seeing exactly which letters a text contains, not just what the words mean.
- How
- Models spell words, say whether a word contains a letter, and insert, delete or swap letters inside words. They read text as tokens rather than letters, so these tasks probe a known blind spot.
- Best AI
- 13.1%
- Humans
- —
- Our items
- 6
- Perception gap, overall
- +13.0
Example item
A specimen written for this page. Live items are similar but never shown, so posted answers can't spoil them.
Read the sentence, then type the number. 30 seconds.
(a) Stimulus
(b) Question
How many times does the letter t appear (capital or small)?
Human vs AI
What the benchmark reports, and where the faculty stands overall
Best published AI
13.1%
best published score at swapping two letters in a word
Humans
—
This benchmark doesn't report a human figure.
Perception overall · faculty reference, 0–100
Measured on other tests, not on CUTE.
Human: Perception Test, untrained adults. AI: Perception Test, Gemini 2.5 Pro.
How we mirror it
Three kinds of item: count how often a letter appears in a sentence; count a word in a short note where one word is repeated across a line break; and find the word whose two neighbouring letters were swapped into another real word, so a spellchecker would not catch it.
Answers are exact counts or a single word, so scoring is automatic. A count that misses the repeated word is logged as a lure, like the decoys in the attention items.
Items are original, versioned, and assigned at random, so a posted answer spoils only one variant.
What we record