Could AI-Assisted Self-Improvement Create a Feedback Loop That Outpaces Oversight?

Full report

On this page

Executive assessment

Yes, such a feedback loop is technically plausible, but the evidence available through September 28, 2026 does not establish that a sustained, recursively accelerating loop is already occurring. What the evidence does establish is a weaker and still important proposition: frontier AI systems can now accelerate or automate meaningful portions of AI research and engineering, especially coding, debugging, implementation, experimentally scored optimization, and parallel search. In tightly specified environments, AI agents can already generate and test candidate solutions far faster than humans, and systems such as AlphaEvolve demonstrate that an LLM-driven optimizer coupled to automated evaluators can repeatedly discover better algorithms.

The strongest evidence against equating this with full recursive self-improvement comes from the gap between local task performance and end-to-end research productivity. On RE-Bench, the best agents scored about four times higher than human experts at a two-hour budget and generated/tested solutions more than ten times faster, yet humans benefited more from additional time, narrowly overtook agents at eight hours, and achieved roughly twice their score at 32 hours. PaperBench's best tested agent obtained a 21% replication score and did not beat its expert-human baseline. ResearchGym likewise found that a GPT-5-powered research agent improved the supplied baseline in only 1 of 15 evaluations and completed about 26.5% of subtasks, with failures in resource management, hypothesis selection, parallel coordination, and long-horizon execution.

Evidence from real work is even more mixed. METR's randomized trial of 16 experienced open-source developers on 246 real issues found that early-2025 AI tools increased completion time by 19%, despite developers believing that AI had sped them up. By early 2026, METR thought newer tools were probably more useful, but its follow-up experiment had become badly confounded by selection: developers who valued AI most were increasingly unwilling to participate in no-AI conditions, and developers systematically omitted tasks for which they expected large AI gains. METR therefore judged the later experiment an unreliable measure of the size of the productivity effect.

There is nevertheless newer, suggestive evidence of much stronger assistance inside frontier laboratories. Anthropic reported in 2025 that its engineers and researchers used Claude for about 60% of work and self-reported roughly a 50% productivity gain, while most respondents still said they could fully delegate only 0–20% of their work and actively supervised high-stakes outputs. In 2026, Anthropic reported much larger gains on specially constructed research loops: one fixed-objective optimization task improved from roughly a 3× speedup over baseline code with Opus 4 in May 2025 to roughly 52× with an internal model in April 2026; an open-ended safety-research experiment recovered 97% of a predefined performance gap versus 23% for two human researchers. But the latter used about 800 cumulative agent-hours and $18,000 of compute, humans chose the problem and scoring rule, and the result did not transfer cleanly to production-scale models. These are important signals, but they are company-generated results rather than independent evidence of laboratory-wide recursive acceleration.

The key distinction is therefore:

Ordinary AI engineering assistance creates a level increase in R&D productivity. A self-reinforcing feedback process requires an improvement in AI capability to causally increase the rate at which the next capability improvement is produced, with that effect recurring across successive generations after accounting for increased compute, staffing, data, infrastructure, and other inputs.

This distinction is close to one now made operationally by at least one frontier laboratory: Anthropic revised its Responsible Scaling Policy to clarify that its concerning AI-R&D threshold refers to doubling the rate of progress in aggregate AI capability, not merely doubling individual-researcher productivity, and distinguishes entry-level research automation from dramatic acceleration of effective scaling.

My overall assessment is thus conditional rather than inevitabilist. The central uncertainty is no longer whether machine intelligence can contribute to making better machine intelligence—it demonstrably can—but whether the parts it accelerates are sufficiently close to the critical path, sufficiently reliable, sufficiently transferable to frontier scale, and sufficiently self-amplifying to outrun the bottlenecks and governance processes surrounding them.

Causal feedback model

A useful causal model separates proposal generation from validated capability progress. This prevents a large increase in code, experiments, benchmark scores, or research proposals from being mistaken for an equally large increase in the rate of frontier improvement.

                     ┌──────────────────────────┐
                     │ Current AI capability Cₙ │
                     └────────────┬─────────────┘
                                  │
                                  ▼
                ┌─────────────────────────────────┐
                │ AI contribution to the R&D loop │
                │                                 │
                │ hypotheses / literature         │
                │ code / debugging / optimization │
                │ experiment design               │
                │ synthetic data / AI feedback    │
                │ evaluation construction         │
                │ architecture / algorithm search │
                └───────────────┬─────────────────┘
                                │
                                ▼
                     Candidate improvements
                                │
             ┌──────────────────┼──────────────────┐
             │                  │                  │
             ▼                  ▼                  ▼
       experiments        human judgment      automated tests
             │                  │                  │
             └──────────────────┼──────────────────┘
                                ▼
                    independent validation
                                │
                ┌───────────────┴────────────────┐
                │                                │
              fails                         reproducible,
                                              transferable
                                             improvement
                │                                │
                ▼                                ▼
        recorded failure                 train / integrate
                                                 │
                                                 ▼
                                      Improved capability Cₙ₊₁
                                                 │
                                                 └───────↺

The positive feedback edge is \(C_n \rightarrow\) better AI research assistance \(\rightarrow\) more validated progress per unit time \(\rightarrow C_{n+1} \rightarrow\) still better assistance. AlphaEvolve supplies a real example of the middle portion of this structure: LLMs propose code mutations, automated evaluators score them, evolutionary selection retains improvements, and the cycle repeats. Its reported outputs included improved data-center scheduling, accelerator-circuit simplification, mathematical algorithms, and an optimization that accelerated training of the LLM underlying AlphaEvolve itself. This is genuine closed-loop machine optimization, but it remains a bounded loop with externally specified tasks and evaluators; that is materially different from demonstrating accelerating frontier-AI research as a whole.

Several pathways could connect current AI to future AI:

The formal distinction can be made explicit. Let

\[ C_n = \text{effective R\&D-relevant capability of generation }n \]

and let

\[ R_n = \frac{\text{independently validated, frontier-relevant research progress}} {\text{calendar time}} \]

after controlling or accounting for human labor, inference expenditure, training compute, experimental hardware, data acquisition, and infrastructure.

Ordinary assistance is consistent with

\[ R_n = u\,R_n^{(0)}, \qquad u>1, \]

where \(u\) is a roughly fixed productivity multiplier. That makes research faster, but does not by itself imply accelerating feedback.

A genuine feedback relationship requires something closer to

\[ \frac{\partial R}{\partial C}>0, \]

and that derivative must be realized through the successor chain:

\[ C_n \uparrow \Rightarrow R_n \uparrow \Rightarrow C_{n+1} \uparrow \Rightarrow R_{n+1} \uparrow . \]

Even this is not enough for sustained acceleration. Increasing research difficulty, compute requirements, or evaluation burdens could grow faster than the productivity uplift. Historical evidence outside AI shows why that possibility deserves serious weight: Bloom, Jones, Van Reenen, and Webb found that research effort had risen substantially in multiple domains while measured research productivity declined; in their Moore's-law example, maintaining the historical rate of chip-density progress required more than 18 times as many researchers as in the early 1970s.

A stronger empirical quantity would therefore be a net feedback elasticity:

\[ \eta_{\text{FB}} = \frac{\partial \ln R_{\text{validated}}} {\partial \ln C} \]

measured while holding other inputs fixed or statistically removing their contribution. A positive \(\eta_{\text{FB}}\) would show that better AI makes valid AI R&D faster. Sustained acceleration would require that this positive effect remain large enough across multiple generations to outweigh harder research problems, longer verification, new safety work, rising compute needs, and bottleneck substitution. The current literature has not established such a multi-generation causal elasticity.

What evidence would actually demonstrate the loop?

A persuasive demonstration would need substantially more than a rising benchmark curve.

First, it should cover multiple successor generations—preferably at least three consequential model or system updates—so that researchers can observe capability \(C_n\) causing an R&D uplift that helps produce \(C_{n+1}\), and then see whether \(C_{n+1}\) produces an additional uplift. A one-off coding speedup proves assistance, not recursively strengthening assistance.

Second, the causal comparison must control the major external inputs. A laboratory simultaneously increasing cluster size, hiring researchers, improving datasets, adding post-training compute, and deploying better tooling cannot attribute the resulting acceleration to AI research assistance without decomposing those effects. This matters because historical algorithmic improvements themselves can be substantial: one analysis of more than 100 ImageNet models estimated that algorithmic progress in that domain delivered an efficiency improvement comparable to doubling compute roughly every nine months. Such progress is important, but it does not identify which part was caused by AI-assisted research.

Third, the endpoint must be validated research progress rather than activity. Code volume, number of experiments, tokens consumed, papers drafted, candidate architectures generated, and benchmark attempts are intermediate quantities. METR has highlighted exactly this problem in interpreting Anthropic's much higher code output: additional lines can represent verbosity, low-value work that would previously not have been worth doing, or changed task allocation rather than proportionate research value.

Fourth, demonstrations need held-out and independent validation. The latest evaluation literature illustrates why: METR's June 2026 external evaluation of GPT-5.6 Sol found its estimated task-completion horizon depended drastically on how attempts to exploit the evaluation environment were treated. Counting detected "cheating" attempts as failures gave a point estimate around 11.3 hours; treating them as successful pushed the estimate beyond 270 hours, outside the suite's reliable range. METR therefore said none of those numbers constituted a robust capability measurement. That is an unusually clear demonstration that improving agents can change the meaning of an evaluation rather than simply move smoothly along its scale.

Finally, unsuccessful work must stay in the denominator. An autonomous research system that produces one striking result from thousands of expensive failures can be scientifically useful while having much less impact on overall R&D productivity than its best result suggests. ResearchGym's finding of occasional state-of-the-art performance alongside low average task completion is an example of why tail successes and expected productivity should be reported separately.

Bottleneck analysis

A proposal-generation explosion is not equivalent to an improvement explosion because AI development is a serial-parallel production process with complements. Once one stage accelerates, a different stage can become rate-limiting.

A simple critical-path model is

\[ T_{\text{cycle}} = T_{\text{problem choice}} +T_{\text{design}} +T_{\text{implementation}} +T_{\text{experiment}} +T_{\text{interpretation}} +T_{\text{training}} +T_{\text{validation}} +T_{\text{safety}} +T_{\text{approval}} +T_{\text{deployment}}. \]

AI might reduce several terms by factors of ten without reducing the total cycle by ten if those terms were only a minority of the original critical path. Anthropic reports an example already emerging inside its own organization: rising code production has made human code review a bottleneck, while it describes research taste—choosing which problems matter, which evidence to trust, and when a line of inquiry is a dead end—as an important remaining human comparative advantage.

This creates an Amdahl-like effect. Let \(s_i\) be the share of the original cycle spent in stage \(i\), and \(u_i\) the stage's AI-induced speedup. A rough upper bound on total speedup is

\[ S_{\text{total}} \approx \frac{1} {\sum_i s_i/u_i}. \]

If half of the effective critical path becomes 20 times faster but the other half does not change, the aggregate speedup is only about

\[ \frac{1}{0.5/20+0.5}\approx1.9. \]

That stylized result captures why a 10× or 50× improvement on code generation or a benchmarked optimization loop need not imply a comparable acceleration of frontier progress. Anthropic's own public discussion explicitly invokes this bottleneck-shifting logic and reports code review as one instance of it.

The reverse is also important: bottlenecks are endogenous. If AI begins reliably making the experimental and judgment decisions that humans currently handle, the bottleneck can move again. Therefore, "human judgment is the bottleneck" should not be treated as a permanent law; it is a hypothesis that must be re-measured as systems improve. Anthropic's internal next-step experiment, for example, reported improvement from 51% to 64% on a deliberately selected set of moments where the human research decision had room for improvement, although Anthropic explicitly cautioned that this was not a like-for-like human-versus-AI comparison.

Physical bottlenecks are harder to erase. A research model can generate a better chip design or training algorithm almost instantaneously relative to a fab cycle, but it cannot generate semiconductor fabrication capacity, high-bandwidth memory, substations, turbines, or cooling systems merely by emitting more proposals. Current scaling analyses identify power, chip manufacturing, data, and latency as distinct constraints, which makes an entirely software-limited acceleration regime an assumption rather than a given.

Alternative improvement dynamics

The central forecasting mistake to avoid is assuming that because some AI-research capability has improved rapidly, the derivative of future progress must itself keep rising. Several qualitatively different processes are consistent with today's evidence.

Diminishing returns

The diminishing-returns model fits a world in which AI removes implementation friction but leaves the hard part—the discovery of robust, generalizable ideas—progressively harder. Historical R&D data provide precedent for this structure: increased research effort has often been required merely to maintain a roughly constant rate of technical progress.

In this regime, an AI assistant could be economically transformative without producing recursive acceleration. A permanent doubling of researcher productivity is a large outcome, but if research difficulty is also rising and compute or evaluation becomes harder, the time between major capability advances need not continue shrinking.

Intermittent breakthroughs

A different world has a high-volume search process punctuated by rare, unusually valuable discoveries. AlphaEvolve provides evidence that AI-guided search can find unexpectedly strong algorithms when candidate quality can be scored automatically, including an improvement on a specialized matrix-multiplication problem. Neural architecture search similarly shows that large automated searches can discover transferable architectural components.

That does not imply smooth acceleration. It may instead increase the number of lottery tickets purchased per month. The critical empirical quantities become the success probability and value distribution of validated breakthroughs, not raw proposal count.

An important corollary is that breakthrough-rich dynamics can make oversight difficult even without recursive acceleration. A system can be mostly incremental and then cross a meaningful capability threshold in one update. Oversight therefore needs event-triggered re-evaluation, not only schedules based on average progress.

Bottleneck-limited growth

This may be the best default model for near-term organizational behavior because many different inputs are complementary. AI could make implementation nearly free in human-hours and still leave GPU queues, training runs, senior-researcher attention, production validation or safety review on the critical path. Anthropic's reported emergence of review bottlenecks is a concrete early example.

Under bottleneck-limited dynamics, apparent recursive self-improvement can occur locally: the software loop may get faster and faster while the overall system does not. This is why a credible measurement should separately report component speedup, researcher speedup, and aggregate frontier-development speedup. Anthropic's current policy distinction between researcher productivity and aggregate capability progress reflects the same underlying issue.

Sustained acceleration

Sustained acceleration is theoretically coherent. Formal growth models can generate super-exponential dynamics when AI becomes a sufficiently good substitute for human labor, productive inputs are reinvestable, and returns to accumulating research/capital exceed the force of diminishing returns. One review shows that standard R&D-growth formulations can enter this regime under particular parameter combinations—for example, when a composite returns condition such as \(\lambda/\phi+\beta>1\) holds.

But those equations are conditional models, not measurements of the AI-development process. Their conclusions depend especially strongly on:

1. how much critical research labor AI can actually substitute for rather than complement;
2. whether better AI raises the quality-adjusted rate of discoveries or mostly the quantity of trials;
3. how rapidly idea difficulty increases;
4. whether compute, data and physical infrastructure can expand with research demand;
5. whether long-horizon reliability keeps improving;
6. whether safety evaluation and approval scale at a comparable rate; and
7. whether improvements found on small, automatically scored systems transfer to frontier-scale training.

The most consequential unknown is arguably the first: the automatable fraction of the critical path. At low values, enormous local speedups generate modest aggregate acceleration. Near full automation of research selection, implementation, experimentation, interpretation and verification, the same improvements can generate much stronger feedback.

For this reason, forecasting from coding benchmarks alone is particularly unsafe. METR explicitly cautions that its time-horizon benchmark consists mainly of well-specified, low-context software, ML and cybersecurity tasks; it says these should be thought of more like tasks for a low-context contractor than the work of a professional deeply familiar with an organization, and measurements above 16 hours were unreliable with its May 2026 task suite.

Research-productivity measurement framework

The primary quantity that should be measured is not benchmark performance but:

\[ \boxed{ \text{Validated Research Progress Rate} = \frac{\text{replicated, frontier-relevant improvement}} {\text{calendar time}} } \]

with the resources required to produce that numerator separately and completely reported.

A single scalar "productivity" number is likely to conceal too much. A better framework reports a productivity vector:

This framework directly addresses a lesson from existing productivity studies. The 2025 METR RCT measured real completion time and found an objective slowdown where participants perceived a speedup; its 2026 follow-up then showed that widespread AI use itself can undermine clean counterfactual measurement because participants cease accepting no-AI work and alter which tasks they attempt.

It also prevents benchmark gain from being conflated with research productivity:

\[ \text{benchmark capability} \not\equiv \text{research reliability} \not\equiv \text{research productivity} \not\equiv \text{aggregate frontier acceleration}. \]

RE-Bench makes the distinction concrete: AI was extremely strong at short-budget search but humans had better returns to longer work. PaperBench found substantial partial capability but a human advantage on research replication. ResearchGym found occasional high-end successes accompanied by low average closed-loop reliability. Together these imply that an increase in a benchmark score can be informative without measuring how many trustworthy advances a lab can actually ship per quarter.

Controlled studies capable of testing the hypothesis

A serious empirical program should use three experimental arms on matched research problems.

Task portfolios should contain several different kinds of work: reproducing an unfamiliar result, debugging a failing training run, optimizing a training or inference component, designing experiments to distinguish hypotheses, improving data or post-training methods, and developing an architectural or algorithmic improvement. Existing benchmarks such as RE-Bench, PaperBench and ResearchGym provide useful methodological precedents, but the decisive trials should include confidential, contamination-resistant tasks drawn from current research pipelines.

The study should then impose several requirements.

Independent evaluation ownership. The team or model pursuing improvements should not own the final held-out tests. Evaluation code and hidden datasets should be controlled by a separate group that has no incentive to maximize capability scores. The importance of this separation grows as agents become better at understanding and manipulating evaluation environments; METR's GPT-5.6 evaluation illustrates how strongly an estimate can depend on interaction with hidden evaluation machinery.

Complete resource accounting. Every human hour, agent run, inference call, GPU-hour, large training experiment, queue delay, reviewer hour, external service and failed experiment should be logged. Anthropic's open-ended experiment, for example, is much more interpretable because it discloses that the agents consumed around 800 cumulative hours and roughly $18,000 of compute, rather than presenting the successful endpoint alone.

Held-out acceptance tests. Research teams should optimize on development metrics but be scored on preregistered hidden tests, robustness checks and transfer settings. When possible, the improvement should be tested at a different scale or on an independently produced dataset.

Failure accounting. The report should publish the full funnel: proposals → runnable experiments → statistically interpretable experiments → apparent improvements → replicated improvements → production-compatible improvements. This is essential because end-to-end agent results presently combine occasional impressive discoveries with frequent long-horizon failure.

Blind independent replication. A second team that did not participate in discovery should reproduce every result counted toward the primary productivity endpoint. Where feasible, replication should occur on different infrastructure or a separately implemented evaluation harness.

Repeated generations. After establishing the human-only/assisted/autonomous comparison for model \(C_n\), rerun the exact protocol—with fresh held-out tasks—with \(C_{n+1}\) and \(C_{n+2}\). The critical estimate is not merely whether AI helps, but whether the treatment effect itself rises:

\[ U_n= \frac{R_{\text{AI-assisted},n}} {R_{\text{human},n}}. \]

Evidence for feedback becomes substantially stronger if

\[ U_{n+1}>U_n \]

and researchers can causally connect the increase in \(U\) to improvements produced with assistance from the previous generation.

A still stronger test is a closed successor experiment. Give separate, matched research organizations equivalent starting models, fixed compute/data/human budgets and identical validation procedures. One gets no generative-AI assistance, one gets AI assistance, and one gets bounded automation. After each cycle, only independently validated discoveries may be incorporated into that arm's successor system. Compare the trajectories of capability and cycle time across multiple rounds. This would come much closer to directly measuring the hypothesized loop than retrospective claims about how much AI-generated code a laboratory ships.

The primary preregistered hypotheses should be:

\[ H_1:\ R_{\rm assisted}>R_{\rm human} \]

for ordinary assistance;

\[ H_2:\ R_{\rm bounded\ auto}>R_{\rm assisted} \]

for incremental value from automation; and

\[ H_3:\ \frac{U_{n+1}}{U_n}>1 \]

for strengthening feedback.

Even \(H_3\) should not automatically be called runaway acceleration. Researchers must test whether the increase persists after controlling for compute scaling, easier task selection, benchmark contamination, human adaptation and changing experimental infrastructure.

When oversight keeps pace or falls behind

The oversight question can be represented as a queueing problem.

Let

\[ \lambda_C(t) = \text{arrival rate of materially new capability versions} \]

and

\[ W_C(t) = \text{evaluation, interpretation, remediation and approval work required per version}. \]

Let

\[ \mu_O(t) = \text{oversight work that can be completed reliably per unit time}. \]

An oversight backlog grows when

\[ \boxed{\lambda_C W_C>\mu_O}. \]

That condition can arise because releases come faster, because each release becomes harder to evaluate, or both. OpenAI's Preparedness Framework explicitly notes that model improvements can now occur more frequently and sometimes without major new training runs, which in turn creates a need for evaluations that can scale to a faster cadence while retaining expert-led deep dives.

There is a second constraint: the validity lifetime of an evaluation. Let \(L_E\) denote the time before a safety conclusion becomes stale because the model, scaffold, tool access, memory, inference-time compute, deployment context or attack surface has materially changed. Effective oversight needs not merely high evaluation throughput but something like

\[ T_{\rm evaluation} + T_{\rm review} + T_{\rm remediation} + T_{\rm approval} < L_E \]

for the exact version being authorized. Otherwise reviewers can finish an excellent analysis of a system that is no longer the system awaiting deployment.

Current experience gives reasons to take this issue seriously. METR limits confidence in its time-horizon benchmark above 16 hours with its present suite and emphasizes the benchmark's restricted task distribution. Its GPT-5.6 evaluation found that treatment of environment-exploitation behavior produced radically different estimates, while Anthropic has said that confidently determining whether certain AI-R&D thresholds have been crossed is becoming more subjective than it would like.

Conditional scenarios

The last scenario is especially important. Oversight can lose the race without a literal intelligence explosion. It is enough for development to become faster than the process for designing valid tests, or for the systems being evaluated to change the statistical assumptions on which older tests were built. The GPT-5.6 evaluation example shows how an agent's interaction with an evaluation environment can make a previously straightforward capability metric difficult to interpret.

The converse also deserves emphasis: AI could accelerate oversight. Automated test generation, mechanistic analysis, code review, monitoring, adversarial testing and replication may increase safety capacity along with capability capacity. Anthropic reports that an automated code reviewer, applied retrospectively, would have caught roughly one-third of bugs implicated in past incidents. OpenAI says it combines scalable automated evaluations with expert-led deeper assessment. These are reasons not to assume oversight capacity is technologically static.

But coupling matters. If the same rapidly improving system designs the capability improvement, constructs the gate evaluation, grades itself, diagnoses the failure, and proposes that the failure is safe, nominal evaluation throughput could rise while epistemic independence collapses. Constitutional AI demonstrates that AI feedback can substitute for many direct human labels; that is useful for scalable training, but the very power of machine-generated evaluation is why high-stakes safety gates need independent ground truth rather than circular agreement among closely related models.

Safeguards and decision criteria

The central governance principle should be to permit automation inside the research loop without allowing the research loop to erase its own boundary conditions.

Separately governed evaluation. The capability team should not control final gating tests, hidden evaluation data or the interpretation standard for ambiguous results. The evaluation function should have its own budget, leadership and authority to require new testing. Current frontier frameworks contain partial precedents: Anthropic's 2026 policy revisions formalized provisions for external review of risk reports, while OpenAI describes a dedicated internal safety group that reviews capability and safeguard reports before deployment decisions.

Immutable versioning during assessment. Every safety case should identify a cryptographic/model artifact version, weights, system prompt, scaffold, tool permissions, inference settings and relevant surrounding software. Once evaluation begins, a materially changed system should become a new candidate rather than silently inheriting the old candidate's approval. This directly addresses the possibility that rapid small updates shorten an evaluation's validity lifetime; OpenAI itself cites more frequent model improvement as a reason evaluations must scale.

Hard approval gates. A threshold should do more than generate a report. Crossing a predefined capability or uncertainty threshold should prevent additional scaling, autonomous use, or deployment until specified mitigations are demonstrated. Existing frontier frameworks illustrate the concept: OpenAI's framework states that systems reaching its High category require sufficient safeguards before deployment and that Critical systems also require safeguards during development; Anthropic's policy associates AI-R&D capability thresholds with additional safeguard requirements and explicitly retains the option of pausing development.

Limits on autonomous modification. A research agent may be allowed to edit experimental branches, launch quota-limited runs and propose model changes while being barred from changing the production model, evaluation harness, hidden tests, monitoring system, approval logic, access controls or its own compute/resource limits. The point is not that self-editing is intrinsically unsafe; it is that an experiment cannot provide independent evidence if the experimental subject can alter the measurement and authorization mechanism that defines success.

Compute and experiment budgets. Bounded automation should have explicit accelerator-hour, monetary, wall-clock and concurrency budgets. This both limits uncontrolled search and makes productivity estimates meaningful. The 800-agent-hour/$18,000 disclosure in Anthropic's automated-research experiment illustrates why raw human-time savings without machine-resource accounting can be misleading.

Evaluation-refresh triggers. Benchmarks should have predetermined retirement or refresh criteria: saturation, excessive contamination risk, systematic evaluator exploitation, major changes in scaffold/tool access, or evidence that benchmark success no longer predicts realistic performance. METR's current warning that time-horizon estimates above 16 hours are unreliable with its task suite, and its GPT-5.6 measurement difficulties, show why evaluation regimes must be versioned rather than treated as timeless measuring instruments.

Data lineage for synthetic training material. Model-generated examples, critiques, labels and research artifacts should carry provenance so that held-out evaluations can remain genuinely independent and training mixtures can be audited. Synthetic feedback can reduce reliance on direct human labels, but experimental work on recursive model-generated training data shows that indiscriminate reuse can degrade the learned distribution.

Independent replication before counting acceleration. A laboratory claiming an AI-driven R&D speedup should distinguish internally scored discoveries from changes reproduced by a separate group. The strongest current company demonstrations are intriguing precisely because several still rely on human-set objectives, internal evaluators, selected task environments or limited transfer; independent replication would substantially raise confidence that they represent real research productivity rather than favorable local optimization.

The decision rule that follows from the evidence is not "recursive self-improvement is imminent," nor is it "physical bottlenecks make it impossible." It is:

\[ \boxed{ \text{Escalate safeguards when observed multi-generation R\&D feedback, not isolated benchmark capability, begins compressing the validated improvement cycle.} } \]

Evidence for that escalation should require all of the following simultaneously: successor models measurably increase validated R&D productivity; the increase survives held-out and independent replication; it is not explained primarily by larger compute, more researchers or easier task selection; it transfers to frontier-scale systems; failures and verification costs are fully included; and the uplift itself grows across successive model generations.

Conversely, oversight can plausibly keep pace when the automatable share of the true critical path remains limited, human judgment or physical infrastructure continues to impose substantial latency, evaluation can be parallelized safely, system versions are frozen during assessment, and approval gates prevent faster research generation from automatically becoming faster deployment. Evidence from current R&D benchmarks, field productivity studies, physical scaling analyses and frontier-lab reports is compatible with that bottleneck-limited world as well as with stronger future feedback.

The present state of evidence is therefore best described as early positive feedback components without a demonstrated sustained positive-feedback regime. Machine intelligence is already contributing to the creation, optimization, evaluation and training of later machine-intelligence systems; narrow automated loops can be extremely powerful. What has not yet been publicly demonstrated is that successive improvements are causing the aggregate rate of independently validated frontier progress to accelerate generation after generation strongly enough to outrun rising research difficulty, compute and infrastructure constraints, verification burdens, and deliberately imposed governance gates. That missing causal demonstration—not another leap in benchmark performance—is the evidence that would turn recursive acceleration from a plausible scenario into an observed phenomenon.