Google says self-improving AI is still far off, internal index shows no research acceleration
Decision Brief
Google DeepMind engineer Haydn Belfield shared internal findings: AI has so far produced no measurable acceleration in AI research itself. AI now writes over 75% of production code, but about 88% of code changes still require human oversight or review, with a human gate before every commit; the share of code changes fully handed to AI rose from 7.5% to 11.9%. The more complex and time-consuming the task, the less likely AI is to complete it independently: across 1389 internal Argon agent runs, success without human intervention dropped sharply as complexity rose. A survey of twelve senior DeepMind researchers also found automation is highest for well-defined, verifiable tasks and lowest for open-ended scientific ideation. On RE-Bench, an internal benchmark of 74 real ML engineering tasks, Argon solved only about half—a big jump over Gemini 3.7 Flash's 27%, but still far from the near-perfect performance full automation would require. The team also built a General Capability Index (GCI) covering 123 models and 92 benchmarks, showing Gemini's capabilities growing steadily at about 0.1 GCI points per month with no significant speedup; Gemini 4 Argon sitting slightly above the trend line is attributed to more compute and organizational investment, not AI-driven acceleration. For teams doing model evaluation and R&D workflow automation, this means keeping humans on creative research, new experiment design, diagnosing subtle training instabilities, and long-term planning in the short to medium term; Google currently rates the risk of AI-driven research automation as low, but is building measurement infrastructure to track whether acceleration signals emerge. This conflicts with the "intelligence explosion" view warned about by Hinton, Bengio and others.
Sources
- The Decoder:AI News
- The Decoder:AI News
留言
登入后即可留言,和其他 builder 交换实测心得。
还没有留言,抢头香。