GPQA Diamond

Rein et al., 2023

View original ↗Epoch AI Benchmarking Hub ↗
FacultyReasoning →

What it measures

A published test we don't mirror: it supplies a reference figure, not test items

GPQA is 448 multiple-choice questions in biology, physics and chemistry, written by domain experts and built to be “Google-proof”: skilled non-experts with unrestricted web access reached only 34%, after more than 30 minutes per question.

Diamond is the hardest subset: the questions experts got right and most non-experts got wrong.

Example question

Written for this page in the style of the test, not one of its questions, so the test set stays unspoiled

Two fermions in a boxspecimen · not scored

Pick one answer. Web search is allowed.

(a) Question

Two identical, non-interacting spin-½ fermions sit in a one-dimensional infinite square well. In the lowest-energy state whose total spin is 1, what is the total energy, in units of E₁ = π²ħ²/2mL²?

  1. (a) 2 E₁
  2. (b) 5 E₁
  3. (c) 8 E₁
  4. (d) 10 E₁
Figure 1. A graduate physics question. Searching the terms doesn't help; knowing the physics does. In the style of GPQA Diamond. Answer: (b). Spin 1 is symmetric, so the spatial part must be antisymmetric: one particle in n = 1, one in n = 2, so 1 + 4 = 5 E₁.

Human vs AI

The faculty-level figures we take from it, 0–100

ReasoningGap -26.7

Humans

69.7

GPQA Diamond, PhD experts

Best published AI

96.4

GPQA Diamond, Gemini 4 Argon (Epoch AI)

These are third-party results on different questions from ours. Where the numbers come from

All reference tests →