Generation

Faculty 02/10 · building blockHuman 50.0 · AI 84.9AI leads -34.92 benchmarks →8 items

At a glance

Producing words, images, sounds or actions, and executing them well, including under constraints.

Generation is producing outputs (words, speech, drawings, actions) and executing them well, including under constraints that must be tracked while producing.

Constraints are where it gets interesting: writing without a given letter forces attention to every word at the level of letters, which is not how fluent production normally works.

Human ref.: 50.0
Best AI: 84.9
Gap: -34.9
Our items: 8

Sub-faculties

The parts of this faculty, what our items cover today, and what is still to come

Sub-faculty	Mirrors	Items	Status
Constrained writing Writing well while obeying a strict rule, such as never using a given letter. Mirrors Constrained writing	Constrained writing	5	Covered
Verbal fluency Producing many fitting words quickly under a rule. Mirrors Verbal fluency task	Verbal fluency task	3	Covered
Images and sounds Producing a drawing or a tune that meets a brief.	—	0	Not covered yet

Human vs AI

Best published result on each side, 0–100

Human ref.

50.0

Best AI

84.9

Humans. Quality varies widely between people, even among professionals. On GDPval, experts judged against other experts set parity by design.

AI. Experts now often rate model deliverables as good as a professional's. Letter-level constraints remain a known weak spot, because models read text as tokens rather than letters.

Human ref. 50.0: The parity line, not a measurement. GDPval's expert graders compare two deliverables blind and pick the better one; a professional's work set against another professional's comes out even, at 50%. About the test →
Best AI 84.9: Share of GDPval tasks where the graders judged GPT-5.5's deliverable as good as or better than a professional's, as reported by OpenAI (April 2026). About the test →

Human referenceOur estimateBest published AILabel = human − AI, points

Example items

Specimens written for this page. Live items are similar but never published.

Name the animalsspecimen · not scored

Type as many different animals as you can in 60 seconds, pressing Enter after each.

(a) Typed in 60 seconds

foxcameleaglecamelunicornsalmon

Figure 1. A category fluency item. Repeats and words outside the category don't count. Benchmark: Verbal fluency task. Answer: Four count here: fox, camel, eagle, salmon. The second camel is a repeat and a unicorn isn't an animal on our list.

Verbal fluency task benchmark page →

Write without Aspecimen · not scored

Write one sentence of exactly seven words that never uses the letter A. 90 seconds.

(a) Task

Write one sentence of exactly seven words that never uses the letter A. 90 seconds.

Response:

Figure 2. A constrained-generation item. Letter-level rules are hard for models that read text as tokens. Benchmark: Constrained writing. Answer: Any valid sentence, e.g. “My old dog sleeps by the fire.”

Constrained writing benchmark page →

Benchmarks

2 rows

Benchmark	Origin	Items	Best AI	Humans
Verbal fluency task	Verbal fluency task (Thurstone, 1938), 1938	3	n/a	—
Constrained writing	Constrained writing (lipogram), 2026	5	n/a	—

In everyday life

Writing to a word limit
Explaining something in simpler words
Following a recipe while adapting it