◆ THE LAB · Models
OpenAI put no test behind its AGI-era claim
The News
OpenAI released GPT-6 Astra, its new flagship model, on September 3, 2026. President Greg Brockman closed the press briefing with “Welcome to the AGI era.” OpenAI’s launch page says the model saturates ARC-AGI-3 at 99.9%; its own benchmark table reports 98.6%. ARC Prize, which built the benchmark, verified 62.7% for the model with a Standard harness and 99.9% with a Provider Adapter harness, and says it is not claiming Astra is AGI. OpenAI’s announcement does not define AGI or state a test that would settle the question.
Chandni Melwani
Founder & Editor
Oct 5, 2026 · 2 MIN READ
Know More
- — The two ARC Prize runs: the Standard-harness test on ARC-AGI-3 Semi-Private cost about $26,000, at Astra’s “max” reasoning setting. The Provider Adapter run cost about $19,000, at “high.”
- — What ARC Prize credits: in the Provider Adapter runs, Astra used 51.7% fewer actions per level than its human baseline on average, and ARC Prize says the model “builds the most precise symbolic model of novel environments we’ve seen.”
- — A third set of figures that does not match: ARC Prize co-founder François Chollet reported 66% on “our standard harness” and nearly 100% with a continuous-conversation harness and custom compaction, at roughly $360 per game. His standard-harness number is not the one ARC Prize’s own write-up publishes.
- — The human baseline: ARC Prize’s human testers were paid $115 per 90-minute session plus $5 per game completed.
- — Safety status: OpenAI says Astra is the first model to reach the Critical cybersecurity level under its Preparedness Framework, and shipped it with monitoring of agentic use and classifiers that halt unauthorised activity.
OpenAI’s benchmark table for GPT-6 Astra puts the model at 98.6% on ARC-AGI-3, and footnotes it: OpenAI ran the test on its own Responses API harness with retained reasoning and compaction switched on, against comparison models on other setups. ARC Prize’s rules allow that. Co-founder François Chollet has drawn the line at harnesses custom-made for the benchmark, and calls general-purpose API settings available to all users fair game.
ARC Prize, which built the benchmark, got two answers from its own verified test. Its Standard harness, which lets a model keep only the notes it writes down, put Astra at 62.7% on the semi-private set. Its Provider Adapter harness, which preserves the model’s opaque reasoning state between requests and applies provider-designed context management, reached 99.9%. Effort differed too, the standard run at Astra’s “max” setting and the adapter run at “high.” ARC Prize reports that across the settings it tested, “max” was the cheapest route to a good score, because harder reasoning solves each game in fewer moves. The bigger number came from the lower setting.
Nobody is disputing the capability. OpenAI had already paused reinforcement-learning training on its latest models intended for deployment over Astra’s cyber capability, and ARC Prize reports that in the Provider Adapter runs Astra used fewer actions than its human baseline on 96.0% of levels. ARC Prize declines the conclusion anyway, writing that it is “not claiming that it is AGI.” OpenAI’s own Charter defines AGI as “highly autonomous systems that outperform humans at most economically valuable work,” and nobody made that claim on September 3.
Asked whether OpenAI was calling Astra the arrival of AGI, Brockman said there is “no contractual AGI triggering anymore, so that’s not a relevant concept.” He called AGI a “mission concept or spiritual concept” and left it to readers to decide for themselves. He also said that for him personally, “I do think we’re there.”
Brockman has most of the contract on his side. Microsoft’s licence once excluded AGI systems altogether; October 2025 moved verification of a declaration to an independent expert panel and extended Microsoft’s rights through 2032 to cover models post-AGI, with appropriate safety guardrails, and April 2026 capped the revenue share and ran it to 2030, “independent of OpenAI’s technology progress.” October left one term keyed to AGI: Microsoft’s rights to OpenAI’s research, the confidential methods used in developing its models and systems, run “until either the expert panel verifies AGI or through 2030, whichever is first.” April’s announcement did not mention it.
Related
Frequently Asked Questions
Who can declare that OpenAI has reached AGI?
OpenAI makes the declaration and an independent expert panel verifies it, under terms the two companies set out in October 2025. Nothing said at a press briefing starts that process.
What else did OpenAI report for Astra?
State-of-the-art results on computer use, browsing, software engineering, cybersecurity, science and professional work. Its benchmark table reports 97.6% on FrontierMath Tier 4 v2, 100% on ExploitBench and 74.1% on DeepSWE v1.1. API pricing is $10 per million input tokens and $50 per million output, with a fast mode at up to twice the speed for twice the price.
Is saturating ARC-AGI-3 evidence of AGI?
ARC Prize says no, and says it made that clear when it launched the benchmark: saturating it would not represent “proof of achieving AGI.” ARC-AGI-3 scores an agent inside novel interactive environments, on levels completed and on actions taken against a human baseline, which is a narrower thing than the economically valuable work OpenAI’s own definition describes.
Sources
- OpenAI, “GPT-6 Astra: A new generation of intelligence” (September 3, 2026) — the saturation claim in the prose, the 98.6% benchmark table, and the harness footnote
- ARC Prize, “OpenAI’s GPT-6 Astra on ARC-AGI-3” — the verified Standard and Provider Adapter results, the reasoning-effort finding, the action-efficiency comparison, and the refusal to call Astra AGI
- ARC Prize results page for GPT-6 Astra — the per-run costs, reasoning settings and human-baseline comparison
- OpenAI, “How enabling two settings tripled our scores on the ARC-AGI-3 benchmark” — what retained reasoning and compaction are, and 13.3% to 38.3% on GPT-5.6 Sol
- François Chollet, ARC Prize co-founder — the rule on which harnesses and API settings are admissible on ARC-AGI-3
- OpenAI Charter — the company’s own definition of AGI
- OpenAI and Microsoft, October 2025 — expert-panel verification, IP rights through 2032 including models post-AGI, and the research-IP term
- OpenAI and Microsoft, April 2026 — revenue share capped and running through 2030, independent of OpenAI’s technology progress
- TechCrunch (September 3, 2026) — Brockman on the contractual status of AGI, and on leaving the judgement to readers
- VentureBeat (September 3, 2026) — Brockman closing the briefing, and his personal view that Astra qualifies
Chandni Melwani
Chandni Melwani is the founder and editor of New in AI, covering AI agents, M&A, and enterprise adoption. She holds a Master's in Management of Artificial Intelligence from Queen's University and brings a practitioner's perspective from her work in Data and AI leadership.
Follow ↗The Lab Briefing
Get the five-beat briefing in your inbox.
The big story, frontier labs, AI at work, e-commerce and the capital crossing borders, each with the part that tells you why it matters, not just that it happened.
Free · No spam · Unsubscribe in one click
More from The Lab
Labs
Anthropic is turning away distillers' paying traffic
Anthropic says early evidence suggests its new defences have cut the data harvested by distillers, who covertly extract Claude's capabilities to replicate them in other models, by around 90 to 99%, and that it has turned away meaningful revenue and trillions of tokens' worth of paying traffic from distillers.
Chandni Melwani · Global · Oct 21, 2026
Models
Meta releases Muse Spark Open under its own licence
Meta's open-weights Muse Spark Open is free of charge under its Muse Model License, but if a company or any of its affiliates offers AI models to third parties via API or a similar interface, and together they gross over $20m in any consecutive 12 months, the company must notify Meta and sign a separate written agreement before offering the model or a derivative as part of that model-as-a-service business. Meta's own table puts it behind GLM 5.3, Kimi K3 and Qwen3.8-2.4T-A95B on the Artificial Analysis Intelligence Index.
Chandni Melwani · Global · Oct 20, 2026
Models
Google's Argon beats Opus 5.5 on LiveBench reasoning, not coding
Gemini 4 Argon costs $2 per million input tokens and $10 per million output through November 30, then $4 and $20, the same as Claude Opus 5.5. On LiveBench 2.0 it beats Opus 5.5 on reasoning but trails it on coding, and it ranks sixth on Vals' Terminal-Bench 4.0.
Chandni Melwani · Global · Oct 18, 2026