Humanity's Last Exam Leaderboard: I Re-Traced the Scores, and the Test Itself Changed
Aug 15, 20266 min readby Jordan Kwan

TL;DR: It depends which leaderboard you mean, and since August there is a new one. On October 22, 2026 I re-ran the trace from the first version of this audit. Scale and the Center for AI Safety have replaced the headline board with HLE-Diamond, a frozen 1,000-question subset, and its leader is GPT-6 Astra at 60.6%. Artificial Analysis, running 2,158 text-only questions, leads with Claude Opus 5.5 at 61.4%. The largest aggregator board leads with Claude Fable 5.1 at 65%, a number that appears on neither primary board, and the aggregator's own provenance note says its table is not sourced from the official leaderboard. Three boards, three leaders, and only two of them ran the test.
What is the Humanity's Last Exam leaderboard, and how many are there now?
More than in August, and they no longer measure the same set of questions.
The stable exam is 2,500 closed-ended questions, finalized on April 3, 2025. Around it now sit three named variants. HLE-Rolling, launched October 8, 2025, is maintained separately and absorbs community corrections. HLE-Verified checks and revises original items. And HLE-Diamond is the new headline: Scale's Diamond page describes 1,000 questions drawn from the existing HLE pool and its held-out reserve after a year of cleaning, split evenly into 500 reasoning and 500 knowledge questions, with frozen question IDs so scores stay comparable across quarters. Scale states the strongest model it tested lands at 60.6%, and that Diamond is where the work lands now.
That one change retires most casual HLE comparisons. A Diamond score and a stable-set score are scores on different tests, and any leaderboard that prints one number per model without naming the variant has hidden the most important fact about it.
What exactly did I check on October 22, 2026?
Four surfaces, in order of authority.
First, the primary boards. Artificial Analysis runs the exam itself: it evaluates models on 2,158 text-only questions from HLE (the full May 2025 revision holds 2,500; multimodal questions are excluded for comparability), scores pass@1, and grades with an LLM equality checker. Its FAQ names its current leader directly. Scale's HLE-Diamond page publishes its main results table with reasoning and knowledge splits, and its page stamp reads updated October 23, 2026 (UTC). Scale's leaderboard index now lists HLE-Diamond, not the original full board.
Second, the CAIS page at agi.safe.ai. A raw fetch on October 22 returns the paper and its author list and no scores at all, because the live board renders client-side. I am therefore not repeating the August CAIS figures as current. They are retired below, dated, not silently carried forward.
Third, the aggregator case: BenchLM's HLE board, 76 rows, updated October 20, 2026.
Fourth, the vendor documents, re-read in full the same day, because the tools question from August turns on them.
Who leads the Humanity's Last Exam leaderboard on each board?
| Board | Variant and size | Leader | Score | Checked |
|---|---|---|---|---|
| Artificial Analysis | Text-only subset, 2,158 questions | Claude Opus 5.5 (Max, Default Fallback) | 61.4% | October 22, 2026 |
| Scale HLE-Diamond | Diamond, 1,000 questions (500 reasoning, 500 knowledge) | GPT-6 Astra | 60.6% (75.6% reasoning, 45.6% knowledge) | October 22, 2026 |
| BenchLM (aggregator) | Stable set claimed, protocol not labeled per row | Claude Fable 5.1 | 65% | October 20, 2026 board |
Scale's Diamond main results, from its own page: GPT-6 Astra 60.6%, Claude Opus 5.5 55.0%, Claude Fable 5.1 51.3%, then a long drop to Gemini 3.8 Flash at 34.3% and GPT-6 Sol at 33.8%. Scale's leaderboard index preview also lists Gemini 4 Argon third at 53.2%, plus or minus 3.07. Artificial Analysis's summary of its own results, dated October 20, 2026, puts Gemini 4 Argon second at 57.1% and GPT-6 Astra third at 54.7% on its text-only run.
BenchLM's podium is Claude Fable 5.1 at 65%, Claude Opus 5 at 64.7%, and Claude Mythos 5 at 64.5%. Two of those three numbers appear on no primary board I could read. Mythos 5 appears on neither Artificial Analysis nor the Diamond board. And BenchLM now says the quiet part on its own page, in a provenance note from its September 22, 2026 editorial check: the official stable-set leaderboard was not the source of its full model table, its source ledger contained provider and secondary reports with no benchmark-owner or independent result rows, and the table does not expose a complete protocol label for every row. That is a better disclosure than most aggregators offer. It is also an admission that the podium is a collage.
Does the tools split still explain the gap?
Yes, and the vendor evidence re-verified cleanly on October 22.
OpenAI's GPT-5.4 mini and nano page, re-read in full, publishes both rows for GPT-5.4 (xhigh): HLE with tool at 52.1% and HLE without tools at 39.8%. Same model, same page, 12.3 points apart. Moonshot's Kimi K2.6 post, also re-read, states in its methodology note that on the text-only subset K2.6 achieves 36.4% without tools and 55.5% with tools, a 19.1-point gap, and it labels its headline HLE section as the full set with tools. Google's Gemini 3.5 Flash model card, published May 19, 2026 and re-read October 22, reports its 40.2% as the full set, text plus multimodal.
The pattern from August holds exactly: vendors state the variant, the subset, and the tools condition. The layer that repackages them still does not, by its own admission. A bare HLE percentage quoted without those three fields is not a measurement. It is a mood, and the wider benchmark habit of replacing a test when scores climb is now visible inside a single benchmark family: when the stable set got crowded, the headline moved to Diamond.
What happened to the August scores?
They are history, and I am treating them that way rather than updating them in place.
In August, BenchLM led with Claude Opus 5 at 64.7%. That exact number is still on its board in October, now in second place under Fable 5.1 at 65%. The August CAIS live-board figures (Claude Fable 5 at 52.72, Claude Opus 5 at 51.0) cannot be re-verified from a raw fetch today, so they are not repeated above as current. The August checks of llmrun.dev, pricepertoken.com, and aitooltier.com were not re-run in October, so their August verdicts, two traceable, one with numbers matching no board and a question count its own cited source contradicts, stand as an August record only. If a claim cannot be re-verified on the day of publication, this page would rather retire it than launder it into the present tense. That is the same standard I apply when checking which model launches publish anything verifiable and when tracking where the labs went after SWE-bench Verified.
What would change my mind?
A protocol column that survives contact with the primary boards. If every quoted HLE number carried four fields, variant, subset, tools on or off, and who ran it, then the three leaders above would stop looking like a contradiction: they are three different measurements, and two of them are honestly labeled at the source. The remaining problem is only the third board, the one most likely to be screenshotted, whose own note concedes it did not take its table from the people who run the test.
Until aggregators print those fields per row, treat a bare HLE score exactly like a claim of PhD-level reasoning: ask which test, under what conditions, graded by whom, and who actually ran it.
Written by Jordan Kwan, founder of Reachium.
I build Reachium, the LinkedIn outreach platform behind the tactics you just read. Same brain, live product.
See what Reachium does ↗Prefer the long version first? What Reachium is and how it works