GAIA

Mialon et al., 2023

What it measures

A published test we don't mirror: it supplies a reference figure, not test items

GAIA asks real-world questions that are simple for people but take several steps: reasoning, reading files and images, browsing the web and using tools.

It has 466 questions; answers to 300 are held back for a leaderboard. Our AI figure comes from Princeton's HAL leaderboard, which runs every agent under the same harness.

Example question

Written for this page in the style of the test, not one of its questions, so the test set stays unspoiled

Sales in dollarsspecimen · not scored

Answer with a single number. You may use the attached file and the web.

(a) Attached file: bakery_2023.xlsx

Q1 2023 €11,900

Q2 2023 €12,400

Q3 2023 €13,100

Q4 2023 €10,800

(b) Question

Using the European Central Bank's average euro to US dollar rate for 2023, what were the bakery's total 2023 sales in US dollars, rounded to the nearest thousand?

Figure 1. A multi-step question: read a file, look up a published figure, then compute. In the style of GAIA. Answer: 52,000. Total €48,200 × about 1.08 dollars per euro.

Human vs AI

The faculty-level figures we take from it, 0–100

Executive functionsGap +17.5

Humans

92.0

GAIA, university-educated testers

Best published AI

74.5

GAIA, Claude Sonnet 4.5 (HAL)

These are third-party results on different questions from ours. Where the numbers come from

All reference tests →