In today’s software development climate, we find ourselves bombarded with AI at every turn, and with conflicting sentiments from everybody involved, it is hard to truly feel like any one person knows what the world will look like, 6, 12 or 24 months from now. But despite all of this noise, I would like to offer some of my own thoughts on the topic.
I’m Scared
Well yeah, it would hard not to be when anyone with half an ounce of power/control is claiming they can make humans obsolete. This as parroted by Microsoft and even our own WiseTech Global founder. As unrealistic as those claims are, they still make even the most optimistic mind fear for the future.
And there are some genuine reasons to be scared, too. With all of the advancements in AI, we are seeing it is becoming clear that AI has managed to fully automate some tasks and even outshine humans in certain regards. This however, is not the level of universal intelligence everybody is hyping it up to be.
AI isn’t all that
Let me preface this by saying: AI is capable, to be sure. It can draw on a wealth of knowledge incomprehensible to most humans and operates at 10x our speed. But it is the finer details that really matter. For this whole section, I will be at various points be referencing Cory Doctorow’s fantastic article Code is a liability (not an asset).
One of the core ideas within Cory’s article is that “Software Engineering” is a far more expansive profession than just “writing code”. So if Richard would like to claim that “writing code” can be done by an LLM, then sure, I may even be (somewhat) inclined to agree. However, this code doesn’t exist in isolation. It is, in fact, in most cases, a small cog in a larger machine or system. And understanding this machine, pre-empting the problems and futureproofing the Software Architecture is the “real” job of a software engineer. Our profession cannot simply be reduced to just “writing code” as it includes much more, from helping the support person answer that question to helping mentor the new associate. And unfortunately for Anthropic, those are all things that cannot simply be “automated” with AI. The context window to understand an entire company just doesn’t exist, and growing the context window requires quadratic growth in compute. It doesn’t have the empathy to teach in any real sense, and that support person certainly won’t be handed a Claude API key with access to the main repository…
So we have addressed how AI just cannot meet most expectations placed upon a software engineer. As sad as it is that so many C-Suite executives just don’t get that, what is even sadder is that they think AI writing code is Creating “Value”. Since as cory so aptly titled his blog article, “Code is a Liability”, and this is true in the most literal sense possible. Among friends, there is a bit of an inside joke that I will blanket approve any PR purely removing lines. And while of course this is quite a bit exaggerated, the deep-seated truth is that more code is always worse. In every way. Comprehension of the code goes down rapidly for both humans as well as LLM’s. Bugs are more numerous, security vulnerabilities more tricky to find and compile times shoot skywards as the compiler has to handle more text and semantics. If manempagement had a simple $ per LOC metric they could whip their employees with, they may realize this themselves. This however isn’t the whole truth because now you need to factor in future proofing, readability and maintainability, a complex balance that every developer strikes decisions about every day while learning from their mistakes of yesterday.
And while LLMs possess the combined knowledge of half the internet, they cannot be novel beyond the most basic of senses. Sure, it can write you a new function name. But it cannot write you a new language with unique semantics. It exists in a state where it is equally able to WOW any executive and flabbergast most Product managers. But pick apart the code, and even the most mediocre developer will find flaws they learned which LLM’s cannot. Since as Cory so elequantly pointed out, “maintainability” is not latent within any codebase but instead constantly changing process-knowledge within the developers themselves.
Are we RoadRunner yet?
It is possibly most scary for me to hear that handwritten code is “artisanal” or in any way special. As most companies transition their workforce to “AI native” or “AI first” mindsets in which 99% of the work is done by AI, the work of preventing critical security vulnerabilities and production outages is growing from a small dedicated team to entire dedicated pipelines / systems. We are trying to build guardrails and train security models to try and keep up with the tsunami of code being written and committed. And since having each AI written PR hand reviewed means slowing the entire PDLC down to the slowest part, the human reviewer (which, btw, nobody wants to do), all of these companies are leaning more and more heavily on AI as part of their review / testing / release deployment cycle.
And in turn, this makes everyone even more dependent on AI and the companies selling these “golden shovels”. And the writing is on the wall. The honeymoon period of cheap AI pricing is over. GitHub has switched to token-based pricing, increasing some users’ bills by up to 52x, Anthropic changed their tokeniser which increased token usage by up to 52%, and ChatGPT released their 5.5 model costing close to 2x that of 5.4.
So, have we done a coyote?
Conclusion
It is hard to imagine that the entire industry is about to implode, and since until that happens everybody is going to keep pushing AI into every nook and crevice it will fit, I suspect we will keep running a little longer. Unsure on whether we are heading towards the cliff or already mid-air, but in any case, I don’t foresee us running these tools at the current scale forever. Especially not with the immense amount of computing power they require to run, which single-handely caused Microsoft to delay/scrap its 2030 clean energy plan. So with a worsening climate, increasing costs and just plain lack of power, it is hard to imagine how much further we can run. But if there is one thing I know, it’s that as long as money is to be made, heaven and earth will be moved to make that happen.
So I’ll keep using it for those problems that it can really help with, and sit by waiting for “I told you so” when all goes to shit, or an “oh damn” if Anthropic announce the Skynet torment nexus that truly replaces our jobs.