LLM coding autopilots still need a steering wheel
(and they will for a long time)
Many experts predicted that autonomous vehicles would be a solved problem by 2020. But, here we are almost 5 years later, and cars still have steering wheels. About a year ago, I had a layover in Phoenix, and experienced the magic of what appeared to be a fully autonomous Waymo ride. This ride was perfectly executed, and felt subjectively better than a human driver. Of course, it was a perfectly clear blue-sky day around mid-day.
It seems that we’ve solved the exciting zero to one problem, where we take something that just wasn’t possible before — car transportation without a steering wheel — and bring it into reality for the first time. But there is a drastic difference between solving the problem in some instances, and solving it for all instances. Now, we’re in for the long slog of increasing the nines of reliability, legal issues, public perception, and the slow, cautious rollout to major cities, minor cities, and everywhere else. At the risk of goal post moving, I’d like to define “success” in the self-driving car problem as when I am legally allowed to sleep and read in a car taking a cross-country (through city) road trip. I’m sure this is many years away (I suspect ~10, and I really hope less than 20).
This type of problem solving / adoption trajectory should be considered the rule, not the exception. Of course, when you are looking for funding, it helps to be optimistic. So when Elon Musk was considering whether or not to put a steering wheel on the $25,000 Tesla, he was adamant that there should not be one. It would take years of work to bring the model to production, and a design decision so fundamental as a steering wheel would certainly add cost and complexity, and would yield an inferior product if a steering wheel was truly not needed. But, Musk’s engineers were able to persuade him otherwise (this is quite rare!). The $25,000 Tesla will have a steering wheel.
Here we are, a decade in to when Tesla first released “Autopilot”, and they still have not committed to actually solving the problem. So what makes you think that companies like Magic.dev or Cognition (Devin) are close to doing it for software engineering? When I speak to my peers about the upcoming wave of automation, there is some fear associated with it. Pretty soon, we might all be out of a job! Even worse is when I speak to new grads or those considering a career in SWE. There is this pervasive fear of automation. It seems like someone has told them the reason why it is so difficult to get a job right now is because of AI. The more probable reason is interest rates — AI happens to be a convenient scapegoat when the market is telling you to reduce force.
Here’s the thing — software engineering is a much harder problem than driving. And I’m not just saying this as a biased luddite or in the sense that Go is harder to solve than Chess, but in the sense that software engineering is a competitive adaptive system, while driving has an optimal solution. Software engineers are always trying to abstract things away, automate, and move up complexity levels. As an adaptive system, it has this interesting property that the more solutions you find, the more the problems expand. A commonly cited example of this is the move from languages which require manual memory management, to languages with garbage collection. Did this help us get to the ‘end’ of software quicker? Did it reduce the demand for software engineers? No - software became faster to produce, so the world demanded much more of it and of much higher apparent quality — more than enough to make up for the time saved by garbage collection. Similar stories include no/low code tools, web page builders, high-level machine learning libraries, etc.
Those companies may have raised money on “automating software engineering”, but my bet is that the main products or value they end up developing in the next 10 years will be small, workflow enhancing tools for human software engineers (but, don’t get me wrong — I believe this alone will be transformative). This is similar to how we’ve seen the fast proliferation of driver assistance technologies such as adaptive cruise control or lane assist, but we haven’t seen an en masse rollout of cars without steering wheels. It’s simply too large of a change for organizations to handle quickly. It would be one thing if the available solution were obviously better in every way, and also fit perfectly as a drop-in replacement to the current solution. But this is almost never the case.
My bet is that these small workflow enhancing tools that make it easier to inject automation and intelligence into software engineering will create hundreds of billions in value before the first AI powered SWE (without a “steering wheel”) gains meaningful adoption. For this reason, I’d bet on tools like Copilot or Cursor, which enhance your current UX rather than completely changing the way you interact with code today.
LLMs have already radically changed the way I approach implementation, and I’ve been experimenting with ways to improve this workflow. To that end, I’ve released an open source command line tool that attempts to solve one of the major annoyances I have with LLM coding workflows. I’ve noticed that I’m comfortable building small projects almost entirely with Claude, but as soon as I start to have too many files, or the chat begins to get too long, it becomes annoying to gather the source files I have in mind to operate on. More annoying than simply performing the implementation myself. The CLI tool, `repogather` makes it easy for me to start up new chats to initiate new features across multiple different files, because I can quickly and intelligently gather all the relevant context the LLM needs, whereas before this would take me a lot of manual copy / pasting to write an instructive prompt.
Check it out here — https://github.com/gr-b/repogather
I would love to hear any feedback, especially if the tool changes the way you approach LLM assisted coding.