Webb, Holyoak & Lu (2023)
View original ↗At a glance
- Tests
- Analogies (Reasoning): seeing that one relationship matches another: A is to B as C is to what?
- How
- Four-term verbal analogies from psychology studies, and letter strings: abc changes to abd, so what does ijk change to?
- Best AI
- 3 of 4
- Humans
- —
- Our items
- 3
- Reasoning gap, overall
- -26.7
Example item
A specimen written for this page. Live items are similar but never shown, so posted answers can't spoil them.
Pick the word that completes the analogy.
(a) Question
- 1. Sculptor is to statue as baker is to …(a) oven(b) bread(c) flour(d) kitchen
Human vs AI
What the benchmark reports, and where the faculty stands overall
Best published AI
3 of 4
verbal analogy sets where GPT-3 matched or beat people
Humans
—
This benchmark doesn't report a human figure.
Reasoning overall · faculty reference, 0–100
Measured on other tests, not on Webb, Holyoak & Lu (2023).
Human: GPQA Diamond, PhD experts. AI: GPQA Diamond, Gemini 4 Argon (Epoch AI).
How we mirror it
New verbal analogies whose wrong options share a surface association with the words (odometer and speed), so only the relationship gives the answer, plus new letter-string analogies.
Each item has two questions; both must be right.
Items are original, versioned, and assigned at random, so a posted answer spoils only one variant.
What we record