Create the Overhang

I first encountered the idea of the overhang from Ethan Mollick.¹ It expresses the idea that there is ample opportunity to exploit AI abilities for rapid progress in many domains using existing tools and techniques. I think this is something that we see in the software engineering profession, where things that used to be tedious but important, like infrastructure as code, are increasingly just a prompt away. I increasingly think that the main contribution of software engineers is to build their own overhang for agents to exploit by creating great primitives and core infrastructure that unlock a plethora of concrete applications for LLMs to exploit.

To see a really stark example, one can look at the recent spurt of math results released by OpenAI in October and the reaction from the math community after they had some time to dig into the details. The background is that OpenAI researchers used internal models to publish 722 manuscripts with new results,² the most notable of which was a result on a new zero-free region for the zeta function. Terence Tao, one of the leading mathematicians in the world, called this the biggest advance on the problem in fifty years,³ reached by an unusual arrangement of three or four existing ideas. Many of the other results had this characteristic of using few new techniques and instead leveraging novel combinations of existing techniques. These techniques had been built up by the mathematical community over decades and were waiting to be used for new purposes, representing a significant overhang.

We see a similar pattern in software engineering. There has been relatively little groundbreaking software designed and built autonomously with AI. As a profession, we have built up a deep set of tools and techniques that let LLMs build incredible things, but this looks more like taking a massive LEGO set and assembling the pieces in novel ways, rather than creating a new form of LEGO. Even with something like their cyberattack capabilities, which have unlocked a new wave of capabilities, what the models are predominantly doing is taking existing known classes of errors, finding new specific implementations of them, and then chaining them together in novel ways. This is largely just a search problem and not a novel design problem.

There was always a tension in software engineering between wanting to focus on building a great core design with beautiful abstractions that would allow things to be built in an elegant way and actually building the thing that mattered. Navigating this tension to build a strong core while being pragmatic enough to actually ship something useful has always been a key part of running a successful engineering org. LLMs really break that tension in a meaningful way. If you can build the great core, the applications fall out of it naturally, and LLMs will execute the grubby assembly easily for you. On the other hand, if you do not have a solid core design that you are building around, LLMs will struggle to build a reliable product that doesn’t require constant hand-holding.

This point really hit home for me over the last couple of weeks while playing around with Pi-durable on the side.⁴ I’ve built a series of small agents to do little tasks over the past two years, but there was also a fair amount of friction in doing it well. With Pi-durable as the core, a working agent with a ton of functionality basically just falls out. You’ve seen an explosion of applications like Mario’s agent that runs fully on his phone with the ability to execute code on any of his machines. A lot of the advanced functionality that other coding harnesses have had to hack together just sort of happens frictionlessly. For me personally, this has been one of the first pieces of software that has come out in the agentic engineering era that has had this effect on what I’m trying to build, and it’s really inspiring to see what good software engineering can unlock.

I think this should be a really inspiring moment for software engineers who value the engineering part of building software. The amount of leverage that you can build with great design is higher than it has ever been. Agents can assist with this design work, but it’s still fundamentally a human task that only a human can do, and it has not gotten any easier. Good systems thinking, understanding how software and computers work, and clear communication are more important than ever, even if we no longer need to memorize the loop syntax in Bash.

Footnotes