Back to timeline

Thu, October 1503:18ResearchModel releasesAgentsInfra & costModel releases guide

Google says self-improving AI is still far off, internal index shows no research acceleration

Decision Brief

What changedGoogle DeepMind engineer Haydn Belfield says AI-driven AI research has shown no measurable acceleration, and self-improving AI remains far off.
Why it mattersThis internal data offers a rare quantitative baseline: AI already writes over 75% of production code but 88% of changes still need human review, and the fully autonomous completion rate rose only from 7.5% to 11.9%—so teams betting on AI as a fully autonomous researcher should expect assistance, not replacement.
Who should careAll AI builders
Affected stackGemini
Source confidenceMedium · Reliable media or first-hand reporting

Google DeepMind engineer Haydn Belfield shared internal findings: AI has so far produced no measurable acceleration in AI research itself. AI now writes over 75% of production code, but about 88% of code changes still require human oversight or review, with a human gate before every commit; the share of code changes fully handed to AI rose from 7.5% to 11.9%. The more complex and time-consuming the task, the less likely AI is to complete it independently: across 1389 internal Argon agent runs, success without human intervention dropped sharply as complexity rose. A survey of twelve senior DeepMind researchers also found automation is highest for well-defined, verifiable tasks and lowest for open-ended scientific ideation. On RE-Bench, an internal benchmark of 74 real ML engineering tasks, Argon solved only about half—a big jump over Gemini 3.7 Flash's 27%, but still far from the near-perfect performance full automation would require. The team also built a General Capability Index (GCI) covering 123 models and 92 benchmarks, showing Gemini's capabilities growing steadily at about 0.1 GCI points per month with no significant speedup; Gemini 4 Argon sitting slightly above the trend line is attributed to more compute and organizational investment, not AI-driven acceleration. For teams doing model evaluation and R&D workflow automation, this means keeping humans on creative research, new experiment design, diagnosing subtle training instabilities, and long-term planning in the short to medium term; Google currently rates the risk of AI-driven research automation as low, but is building measurement infrastructure to track whether acceleration signals emerge. This conflicts with the "intelligence explosion" view warned about by Hinton, Bengio and others.

Summary basis: full article readCompiled from the source scope noted above; the original remains authoritative.

Sources

Related intel

留言

登入后即可留言,和其他 builder 交换实测心得。

还没有留言,抢头香。