Where the test comes from
On 25 October 2024, Simon Willison asked a set of LLMs to “Generate an SVG of a pelican riding a bicycle.” The results were funny, revealing and easy to compare, and the prompt became an informal benchmark. It tests spatial reasoning and code generation through a task that is trivial to state and surprisingly hard to satisfy.
Our variant
We kept the core task and changed the deliverable: a self-contained HTML page with a 2D SVG animation of a pelican riding a bicycle. The animation raises the bar. A static drawing can hide a bicycle that never pedals; a loop exposes the contact points and the motion chain.
Protocol
- One prompt, identical text, no follow-ups and no corrections.
- One run per model. No cherry-picking, no regenerations.
- Consumer clients, default settings, tested on 27 October 2026.
- Outputs saved as self-contained HTML. No external network requests were present in the eight artifacts.
- Outputs are displayed unmodified. Only the wrapper around them is ours.
- Scores are editorial, produced by a human against the rubric below. They are not automated measurements.
The rubric
Each dimension is scored 0–5. The total is out of 25.
| Dimension | What we look for | Points |
|---|---|---|
| Semantics | Is it unmistakably a pelican riding a bicycle — not a duck, a stork, or two unrelated drawings? | 5 |
| Anatomy & contact | Feet on the pedals, wing or body connected to the handlebar, body settled on the saddle. | 5 |
| Bicycle structure | Two wheels, frame, fork, crank, chain, saddle and bars — the parts must form a rideable whole. | 5 |
| Motion physics | Wheels turn in the direction of travel, the crank rotates, feet track the pedals, nothing slides or detaches. | 5 |
| Polish & interaction | Art direction, scene detail, a complete loop, and controls that let you pause or restart. | 5 |
Limitations
- Single run per model: not statistically significant, and vulnerable to random variation.
- Consumer client versions are moving targets. A score is a snapshot of one date, not a permanent property of a model.
- Aesthetic judgment is subjective; the rubric makes the criteria explicit but not objective.
- This is not a general intelligence benchmark. It is a visual smoke test for spatial composition and motion.
- The canonical pelican test is a static SVG. This is an animated extension, and both can be useful.
Attribution
The pelican-on-a-bicycle test was created by Simon Willison. Read his original post, the GitHub repository of model outputs and the 2025 talk-turned-article. For the 2026 “pelicanmaxxing” study of 1,000+ SVGs, see Dylan Castillo’s analysis. The eight HTML files on this site were generated by the named models and are displayed unmodified.
Changelog
- 27 Oct 2026 — v1: eight runs, bilingual English/Chinese site, animated scorecard and sync comparison.