What is the JASON test?

It is the benchmark I give every new frontier AI model: the same CV and written brief, one attempt with no follow-up prompts, inside the model maker's own coding tool, with time, tokens and cost read from the logs. Every site it builds is live.

Which model has been cheapest so far?

Gemini 3.8 Flash, in round 1: $1.48 for a complete working site, in 17 min. It also came back weakest of the three in that round.

Which model built the best site?

In round 1, Fable 5.1 was the clear winner on a one-shot web build, and the most expensive of the three. Round 2 names no winner on design: its sites are live to compare.

Can I compare token counts between rounds?

No. The rounds counted tokens differently: round 1, effective tokens, cached re-reads discounted; round 2, tokens, every token processed. Compare cost and time across rounds instead.

A Build Notes series

The JASON test.

Every time a frontier AI model launches, I give it the same job: my CV and one written brief, build the website, one attempt. What each run took, what it cost, and the site it built, live.

How does the JASON test work?

The same input every time: my CV and one written brief.
One attempt. No follow-up prompts, no fixes, no second tries.
Each model runs inside its own maker's coding tool.
Time, tokens and cost are read from the logs, never from memory.
Every site it builds is published, so you can judge the result yourself.

Round 2 · 23 September 2026

GPT-6 Astra and Opus 5.5

Model	Run in	Time	Tokens, every token processed	Cost	The site
Opus 5.5Anthropic	Claude Code	24 min	14,935,920	$6.89	See it live
GPT-6 AstraOpenAI	Codex	36 min 29 s	4,962,595	$13.38	See it live

Opus 5.5 was faster and cheaper. GPT-6 Astra used a third of the tokens but cost about twice as much, because each of its tokens costs more. No winner is declared on design: both sites are live to compare.

Astra ran in a Codex task that already held earlier portfolio work, so it was not a clean session. Opus 5.5 started from a fresh agent.

At list price from OpenAI's and Anthropic's pricing pages, checked 24 September 2026, with cached tokens at the cached rate. Both builds passed lint and build when re-run.

Read the full write-up →

Round 1 · 3 September 2026

Fable 5.1, GPT-5.6 Sol and Gemini 3.8 Flash

Model	Run in	Time	Effective tokens, cached re-reads discounted	Cost	The site
Fable 5.1Anthropic	Claude Code	47 min	about 11M	$23.57	See it live
GPT-5.6 SolOpenAI	Codex	29 min	about 2.6M	$10.20	See it live
Gemini 3.8 FlashGoogle	Antigravity	17 min	about 1.8M	$1.48	See it live

On a one-shot prompt for web development, Fable 5.1 was the clear winner, and also the most expensive, the hungriest on tokens and the slowest. GPT-5.6 Sol did better on interface and experience. Gemini 3.8 Flash came back weakest, at a fifteenth of the price.

Metered by each tool where shown, otherwise at list API prices.

Read the full write-up →

Common questions

What is the JASON test?: It is the benchmark I give every new frontier AI model: the same CV and written brief, one attempt with no follow-up prompts, inside the model maker's own coding tool, with time, tokens and cost read from the logs. Every site it builds is live.
Which model has been cheapest so far?: Gemini 3.8 Flash, in round 1: $1.48 for a complete working site, in 17 min. It also came back weakest of the three in that round.
Which model built the best site?: In round 1, Fable 5.1 was the clear winner on a one-shot web build, and the most expensive of the three. Round 2 names no winner on design: its sites are live to compare.
Can I compare token counts between rounds?: No. The rounds counted tokens differently: round 1, effective tokens, cached re-reads discounted; round 2, tokens, every token processed. Compare cost and time across rounds instead.

Get the next round

The next model launch gets the same brief, and the results go to Build Notes first. Subscribe and get the free course, Build Your First AI Operating System.