METR questions Anthropic’s acceleration assessment

METR’s review considers it plausible that Anthropic crossed its research-acceleration warning threshold by July 15. Most relevant researchers at both organizations nevertheless lean toward it not having been crossed. METR agrees that the direct catastrophic risks from models available then were low. Its criticism concerns how confidently Anthropic’s public report published in August this year dismissed the warning: the reviewers say it understated evidence for acceleration and uncertainty. Crossing an early-warning threshold would not itself establish that the feared catastrophic harm had materialized.

The separate October assessment examines how AI helps develop better AI. METR describes assistance to researchers as the main contribution, rather than fully autonomous research. Historical and current growth can be observed, but growth without AI is an estimate. Faster benchmark improvement need not translate directly into real-world impact. The team received mediated access to internal data, without requesting or receiving access to sample unreleased models. Anthropic redacted important evidence, limiting public scrutiny of the conclusions. This engagement was separate from the later project promising employee-like access, so those broader access terms do not describe this assessment.

The Decoder.