THE NOISE

AI Is Starting to Build Itself. Here's What the Labs Actually Said.

2026-09-28

Two of the biggest AI companies published essays this month that say, in different words, the same thing: AI is increasingly helping build the next version of AI, and nobody is fully ready for where that leads. That's easy to hype and easy to dismiss. We read both, plus one very public pushback, so you don't have to.

The term to know is recursive self-improvement, usually shortened to RSI. It means an AI system good enough to design and build its own successor, which then builds a better one, and so on. For years it lived mostly in science fiction and research papers. What's new is that the companies building frontier AI are now describing early versions of it happening inside their own walls.

Anthropic, the company behind Claude, made the most data-heavy case. It says that as of May 2026, more than 80% of the code merged into its codebase was written by Claude, up from low single digits before its coding tool launched in early 2025. It also says the typical engineer was merging about eight times as much code per day as in 2024, while admitting that lines of code overstate the real productivity gain, since they measure quantity rather than quality.

The more striking numbers are about research, not code. In one experiment, Claude-powered agents working on an open AI safety problem closed 97% of a performance gap, while two human researchers working for about a week closed roughly 23%. The caveats matter: humans still chose the problem and built the scoring system, and the result didn't transfer cleanly to full-scale models. Outside benchmarks point the same direction, with the length of tasks AI can reliably finish on its own doubling roughly every four months.

A few weeks earlier, OpenAI's chief scientist Jakub Pachocki published an essay called An Alien Mind. His core argument is that modern AI is grown more than designed: it comes from repeating a simple training step on enormous amounts of computing power until something intelligent emerges. Studying it, he says, is closer to neuroscience than engineering. Researchers find small mechanisms inside, but the whole system resists a full explanation.

He's direct about where he thinks this is heading. Based on internal results, he expects the current pace of progress could carry through to recursive self-improvement. He also flags a real worry: one of OpenAI's main safety checks, reading a model's step-by-step reasoning to catch bad behavior, is getting less reliable as models get better at reasoning without writing their thinking down.

Strip away the branding and the overlap between the two labs is remarkable. Both say AI is already speeding up its own development, not in some future scenario but now. Both say humans still steer, with the remaining human edge being judgment: choosing which problems matter, which results to trust, and when an approach is a dead end. And both say they'd want the option to slow down. OpenAI's essay says no lab has solved safety well enough to keep scaling at maximum speed much longer. Anthropic says it would expect to slow down or pause if other frontier developers did the same in a way that could be verified.

Not everyone thinks this is a turning point. Asked about the two essays on The Ezra Klein Show, NVIDIA CEO Jensen Huang pushed back on the drama. His point was that recursive self-improvement is simply how computing has always worked: NVIDIA uses software to design chips, those chips run better software, and that software designs the next generation of chips. That is called computer engineering, he said, and they've been doing it for a long time.

He described AI agents the same way. An agent tries a task, reviews which approach worked best, saves that approach as a reusable skill or memory, and does better the next time. Feed enough of those lessons back in and you can train a better next model. Framed that way, it sounds less like a runaway machine and more like a very fast version of how good teams already improve.

Both views can be true at once. Huang is right that tools building better tools is an old pattern, and anyone who spent years in enterprise software watched exactly that loop play out across every product generation. The labs' warning is about two things that are genuinely different this time: speed, and who's doing the steering. The loop is getting faster, and humans are doing a smaller share of each turn of it. That difference is what this whole debate is really about.

A few honest caveats before anyone panics or celebrates. Anthropic itself writes that it is not there yet and that recursive self-improvement is not inevitable. Exponential curves sometimes flatten, and progress could run into limits on chips, electricity, or the need for a genuinely new idea rather than more scale. These are also the companies building the technology, with every reason to make their tools sound powerful, so treat the numbers as self-reported. And a smarter AI lab doesn't instantly change your week: more intelligence can't speed up how long it takes to learn what a drug does over years, or how quickly organizations adapt.

So what does it mean if you use AI tools for work or a side business? Expect the tools to keep getting better quickly, which is a good reason to avoid long annual plans for anything that could be outpaced in six months. Expect your value to keep shifting from doing the work to judging it, which is the same shift the labs describe inside their own teams. And keep checking the output. If the companies building these systems are still working out how to verify what their own models do, the rest of us should assume we need to verify too.

AI building AI is no longer science fiction, but it isn't magic either. It's a trend worth watching with clear eyes, which is what we'll keep doing here.

Sources: Anthropic Institute, When AI builds itself. Jakub Pachocki, An Alien Mind, OpenAI, September 6, 2026. Jensen Huang on The Ezra Klein Show.