Examining Recursive Self-Improvement
How AI is being used to build AI today and why we’re already facing risks, even though truly autonomous self-improvement is not possible yet
As AI models have gotten “smarter” with each release, they are now handling more complex coding tasks. Besides accelerating development cycles, this is also redefining how powerful AI solutions can be. Therefore, there’s been a growing interest in recursive self-improvement (RSI), which refers to AI’s ability to enhance its own intelligence.
While AI has been incredibly useful in development, we’re still far from complete self-improvement. However, partial self-improvement already exists, and it’s proven incredibly valuable for many companies. As AI handles more of its own development, it will change in ways that affect us all. We need to understand this shift because it will have far-reaching consequences.
What Is RSI?
Recursive Self-Improvement (RSI) is a process where AI systems rewrite and redesign their own underlying intelligence. When AI labs mention this term, they are primarily talking about it in the context of engineering and research work, not about AI’s ability to independently get better at doing things in general. Model developers made a “deliberate choice“ to first focus on making models great at these types of tasks because this would allow them to speed up AI development. In theory, RSI would allow a model to identify flaws and inefficiencies in its architecture, plan improvements to enhance its capabilities, and then implement changes to create a “smarter” version of itself. The improved version could improve itself even more efficiently, creating a self-reinforcing loop.
While the general idea of RSI is easy to understand, there is no consensus around its precise definition, even amongst AI experts. Some define it as a fully autonomous loop, while others define it as any use of tech to build tech. According to stricter definitions, it represents the ability to autonomously improve both its process and its outputs without human involvement. It’s often more useful to think of RSI in terms of specific use cases: Can AI continuously improve its performance on a specific task? For example, code review is a critical bottleneck in many development workflows. If an AI system can continuously and reliably refine its ability to review code, without any human involvement, that is functionally RSI, even if it’s limited to a specific codebase.
What RSI Isn’t
RSI isn’t behavioral-level self-correction.
Many current AI systems can use feedback loops to plan tasks, review intermediate results, and readjust their plans. While this may seem similar to RSI, the system is still following rules established by developers. This apparent “continuous improvement” is the result of deliberately designing constraints within the architecture, not a purely autonomous process.
RSI isn’t AGI or superintelligence.
Current AI systems may exceed human capability in certain areas, but they certainly can’t do everything a person can. Artificial General Intelligence (AGI) represents the threshold where AI can fully match human capabilities. Artificial Superintelligence (ASI) or superintelligence represents AI that exceeds our capabilities. While RSI will likely help researchers work towards AGI and superintelligence, getting there will require new ideas, not just new code optimizations.
What RSI Looks Like Right Now
Building a frontier model requires extensive efforts across engineering and research. The engineering work involves writing code, training models, and building AI infrastructure, while the research involves designing experiments, interpreting results, and figuring out what to try next. AI models are currently helping with work in both areas, so a form of RSI already exists.
What we currently have is called weak recursive self-improvement. In this version of RSI, AI isn’t autonomously redesigning itself, but it’s meaningfully accelerating human efforts to redesign the system. As AI is now used extensively in development, the share of work people do themselves is shrinking in many cases. Developers and researchers are increasingly delegating AI development work to AI systems themselves. They are still heavily involved in critical decisions (e.g., setting goals, defining success criteria, assessing tradeoffs), even if AI handles the majority of execution.
However, AI’s capabilities are continuing to improve, making it possible to delegate more work to it. AI’s coding performance and long-horizon task performance have increased, making it more feasible to use it for the type of complex work needed to enhance AI systems. At Anthropic, engineers now ship ~8x as much code per quarter as they did from 2021 to 2025. As of May 2026, Claude wrote more than 80% of their merged code. Back in February 2025, this number was in the low single digits. It’s likely that other leading AI labs are experiencing similar productivity gains. All this has had compounding effects on AI development, allowing labs to actively use their existing models to build the newer models. For example, OpenAI stated that its Codex model was “instrumental in creating itself.“
There’s growing confidence in the vision of AI that builds itself, without requiring any human input or intervention, assuming foundational models’ coding capabilities continue to improve at their current rate. It’s not surprising that leading AI companies believe that by the end of 2026, self-specifying, self-creating, self-shipping software will largely be here. Even if this vision fails to fully manifest, if labs can make significant progress towards it, that’s still incredibly valuable to companies. Therefore, for practical purposes, the question isn’t “whether self-improvement exists in some form today, but how much of the loop has actually been closed.“
Why Is The AI Industry So Interested In RSI?
Even weak RSI has already proven incredibly useful and commercially valuable. A closed, self-improving loop represents a tipping point beyond which technology could “evolve” completely autonomously. AI companies could use this to significantly enhance their ability to build more powerful systems, unlocking new revenue opportunities, which some of them may desperately need. Currently, despite the massive investments already made in AI, many companies are still struggling to generate proportional returns. Some may be making more money, but they are not making enough to justify their spending.
Since coding-related use cases already account for ~55% of corporate AI spending, it’s understandable that companies want to maximize the returns on this. RSI could reduce iteration costs, compress development timelines, and compound capability improvements in ways that would normally be impossible or at least unfeasible, even with existing AI. This will likely further enhance AI-driven productivity and profitability, making it an attractive prospect for many companies. Since there’s a strong business case for RSI, leading AI labs have a powerful financial incentive to actively pursue it.
Once AI systems can manage the upgrade cycle better than humans, the process can become a closed loop, limited only by the compute power they can access, and humans are no longer necessary or even helpful.
— Russell Brandom, RSI is the new AGI — and it’s just as hard to pin down
Why True RSI Is Harder Than It Seems
Human judgment still matters in many critical tasks, even as AI gets smarter.
AI researcher Nathan Lambert recently wrote about how we should expect “lossy self-improvement (LSI),” instead of true RSI. Generally, RSI requires key assumptions to be true: the loop is closed, self-amplifying, and efficient. Ensuring that all these conditions are met is much more challenging than AI labs like to admit. The AI development process could become substantially automated, but humans may still be steering the process. The small fraction of the work they still handle may be the most important. These are the decisions requiring human taste, judgment, and expertise: which goals to target, which problems matter, which results are good and trustworthy, and which solutions are worth pursuing.
Frontier models still lag behind human experts in direction-setting work, such as managing week-long, ambiguous tasks or understanding organizational priorities. They still don’t exercise effective judgment when choosing engineering and research goals. They are merely decent at generating, implementing, and judging ideas. AI models have jagged intelligence, so they are smart in different ways than humans. The tacit knowledge essential for handling many complex tasks often can’t be seamlessly transferred to AI. Without this, AI is always operating with incomplete context. The version of RSI that many AI companies keep talking about is the one that operates completely independently. But as even Google CEO Sundar Pichai has said when asked about RSI, we “aren’t quite there yet.”
The real bottlenecks are deciding and specifying what to build, verifying what has been delivered, and the deep human understanding of the application, the customers, and the business. Recent research found that AI agents led to an 8x increase in the number of lines of code written, but only 30% increase in releases, strongly suggesting that these bottlenecks remain in place. Even if AI becomes capable of automating 99% of software work, the 1% that still requires human expertise remains a key speed limiter. AI increases the leverage of human judgment, but it still can’t fully replicate or replace it in critical areas. While the exact timeline may vary, in many domains outside coding, RSI may take longer to emerge than expected.
AI development will remain constrained by financial and physical limits.
What’s possible is different from what’s feasible. Even if RSI allows AI systems to continually improve, it will likely require a significant amount of compute and extensive use of the most advanced, most expensive models. While using AI to build AI sounds great on paper, as many companies discovered with the recent Tokenmaxxing trend, this could create unsustainable costs. Major companies and well-funded startups may be able to absorb these costs to an extent, but it likely won’t be feasible for many. As a result, AI labs may struggle to generate the revenue needed to fuel further development of self-improvement capabilities. Many of their customers may not be able to afford to use AI at the scale they need. This may make moving beyond weak RSI challenging as well.
AI labs will have to balance acquiring substantial capital, generating revenue, and spending an extreme amount on research and compute. Even as models improve, this will continue to be a source of friction, slowing down development. Anthropic CEO Dario Amodei has stated that realistically, it’s not possible to afford the compute necessary to keep using AI at the level needed. Even if they can somehow afford it, there may simply not be enough available capacity. AI infrastructure takes time to build and is subject to real-world limits (e.g., resource shortages, construction delays, insufficient energy supply). Expanding data center buildouts, chip fabrication, grid expansion, and bandwidth are often still challenging, even with extensive capital investment and strong political support. In some cases, geopolitical tensions may create unavoidable supply chain issues, which may make infrastructure expansion much slower and more expensive.
What Are The Critical Risks
RSI creates distinct risks because it increases the rate at which AI systems become more powerful, even if the loop isn’t fully closed. The timeline for risks to fully manifest will differ depending on the specific domain, but it’s important to recognize that even progress toward true RSI may have potentially severe consequences for the world.
Maintaining control and oversight gets more challenging as the systems get smarter.
When the system evolves faster than humans can track, architectural changes become less understandable and more challenging to examine. This makes it harder to know how the system is changing and why it’s making certain changes. The models powering these systems can already exploit loopholes and even fake alignment. Together, these qualities already make it challenging for researchers and developers to implement robust constraints and guardrails to restrict unwanted behaviors. RSI would make it much harder to implement measures that will truly be effective because AI could develop in ways we can’t understand or control.
As we’ve trained more capable frontier reasoning models, we’ve found that they have become increasingly adept at exploiting flaws in their tasks and misspecifications in their reward functions, resulting in models that can perform complex reward hacks in coding tasks.
— OpenAI, Detecting misbehavior in frontier reasoning models
Many safety, security, and reliability mechanisms designed to control the system’s behavior may become ineffective, since RSI could enhance AI’s ability to circumvent these. The system could literally build workarounds to get past restrictions. Misalignment would also likely get harder to detect and fix because AI systems could seem aligned even under more rigorous testing. Existing issues could silently compound, leading to problems becoming more frequent and less understandable. Since more capable AI systems will typically handle more consequential work, the fallout from these problems also becomes much more dangerous.
Effectively governing AI requires unprecedented levels of coordination and cooperation.
The institutional regulations, policies, and standards have always lagged behind the frontier because of the nature of bureaucracy and democratic processes. Both public and private institutions need time to understand, evaluate, and deliberate over new technologies. RSI will make the gap between the current state of AI and the governance mechanisms much wider, much faster. Institutions can’t effectively govern such rapidly evolving technology. Many of them don’t seem to have a solid plan to keep pace with AI development, and the AI labs driving the ecosystem don’t seem willing to slow down for them either.
Competitive pressure will also likely decrease support for any meaningful slowdown since both companies and nations are incredibly concerned about “falling behind” in the AI race. In the absence of any global coordination mechanism, the AI industry will continue to prioritize building more powerful AI over addressing the potential problems and risks. A slowdown would require rival companies and governments to accept the same limits, which seems highly improbable. Even if such an agreement were possible on paper, since most AI development happens “outside the public eye,” there’s no way to verify that AI companies are not internally working on RSI to enhance their own systems.
Why The Risks Won’t Slow Down AI Development And Why This Matters
AI labs keep warning people about the potential dangers of powerful AI systems, but they keep accelerating their development efforts anyway. AI labs and their investors have bet enormous sums on this technology’s success. They can’t generate returns at the scale they need without market dominance. Therefore, “winning” the race has become a primary focus, intensifying competition between AI labs. As AI becomes a national priority, geopolitical pressure also increases the incentives for these labs to accelerate their efforts. Dario Amodei (Anthropic’s CEO) has stated that RSI may give whoever leads the AI race a “runaway advantage,” increasing their lead and making it difficult for everyone else to catch up. This is precisely why labs keep working towards complete self-improvement, despite the risks powerful AI systems pose to society.
What makes RSI uniquely concerning is that, in addition to making control and oversight more challenging, it also makes it possible to rapidly enhance systems used for harmful purposes. This threat compounds with each model release. Anthropic’s latest model, Claude Mythos Preview, found thousands of high-severity vulnerabilities in major software programs, sparking concerns worldwide about the potential damage these systems could cause. We’ll likely see similar capabilities from other frontier models very soon. While the response to managing potential threats has been to limit access to models, as we get closer to full RSI, bad actors could enhance their attack capabilities even without access to the latest models.
AI will likely be genuinely helpful across many domains, and RSI may help us realize many benefits much faster. However, AI labs are focused on capitalizing on this technology as much as possible, which is why they are interested in enhancing AI’s “general” intelligence, rather than enhancing its ability to solve the toughest societal problems specifically. While they may be currently focused on achieving true RSI in engineering and research work, the progress they make will have powerful ripple effects across society because technology is so deeply embedded in our lives. In many cases, this may harm people more than it helps them. We need to pay attention to RSI now while it’s still incomplete because once the self-improvement loop is fully closed, things will change at a significantly faster rate, which will be a major problem for all of us.
If you’re interested in learning more about how the reckless competition to build more powerful AI will affect society, check out this post:
Conclusion
AI labs want true RSI because it may unlock new revenue streams that would eventually create the massive returns these labs and their investors are aiming for. That payoff depends on improving AI’s general performance, which will happen as a downstream consequence of RSI. However, this will also amplify the problematic aspects of this technology, which AI companies have already been unable to solve. These problems may remain an issue or get worse over time, and the new ones will likely pop up as well, even as the models get “smarter.”
Implementing robust oversight, control, and governance is fundamentally more challenging in a system that can keep evolving, which is exactly why AI labs themselves can’t fully predict the scope and scale of disruption that rapidly improving models may cause in people’s lives. We can’t assume that all of the “improvements” due to RSI will be positive. We must pay close attention to the AI industry’s efforts to enhance AI’s self-improvement capabilities because if development outpaces companies’ ability to control and constrain emerging capabilities, AI will disrupt our lives much faster than we may be prepared for.
Thanks For Reading
If you haven’t already, please consider subscribing to this newsletter. I hope you have a great week!
Key References
Priank’s Newsletter
Forbes | The Most Important Idea In AI: Recursive Self Improvement
The New York Times | Notable Researchers Join $4 Billion Effort to Build Self-Improving A.I.
Scientific American | Anthropic warns AI may soon begin recursive self-improvement
TechCrunch | RSI is the new AGI — and it’s just as hard to pin down