If you’re not in tech you might not realize just how much of a panic has set in among the C-suite and investor class in the industry. Everyone is, of course, putting on a brave face, but as fears of a SaaS-pocalypse persist and a fictional financial analysis on Substack caused a real sell-off in markets, it’s hard not to see the sweat on everyone’s brows.
Meanwhile, CEOs and CTOs are falling over themselves trying to position themselves as visionaries, with the likes of Jack Dorsey, Sebastian Siemiatkowski, and Matt Biilmann each trying to out-AI-pill one another. Of course, one has to wonder how much of this is AI-washing and how much of it is intentionally eye-catching hyperbole, but these are just the most extreme and visible tip of a very large iceberg.
It’s hard not to enjoy the bitter irony, here. The tech industry once revelled in disruption. Whether it was Uber wiping out the taxi industry, Amazon destroying traditional bookstores, or Spotify wiping out artist revenues, time and time again we’ve seen the Silicon Valley enrich themselves by disrupting an established market and then transforming into rent seekers.
AI now threatens to do the same to countless professions. Everyone from writers to graphic designers to financial analysts can feel the wolf at the door.
But this time things are a little different.
This time it’s coming for tech first.
Anyone who’s working with large language models knows that, despite being incredibly powerful, they are also deeply (and in some ways fundamentally) flawed.
Take, for example, an incredible common use case: summarizing meeting notes.
I’ve used the latest, greatest, and most sophisticated models to perform this incredibly mundane task, and I’d put them at a solid 90% accuracy.
But that 10% definitely matters.
It’s entirely normal for the AI to incorrectly attribute one person’s words with another; to make claims or draw conclusions that aren’t evident from the meeting transcript; to inflate or exaggerate statements or claims in the meeting. The list goes on.
The challenge with AI in these contexts is that there’s no objective source of truth for determining what “correct” is. In the industry, the term for this would be an “oracle”, something that you can use to interrogate your conclusions to assess validity.
The same is true across a wide variety of domains where you might apply AI. Whether it’s writing prose, or generating an image or video clip, or offering a customer a resolution an automated support call, “correct” is often a matter of taste, opinion, or interpretation. As a consequence, application of AI is incredibly uneven across industries.
Moreover, the very nature of large language models and how they’re trained can increase the odds they will be unsuccessful in a variety of domains. LLMs are, after all, ultimately trained on large quantities of publicly available data, and particularly for deep niches, much of what makes and expert an expert isn’t available for easy scraping online.
And then there’s tech.
First, assuming a decent spec or set of requirements (yes, that’s a big if!), software is fundamentally verifiable: either the code does what it’s supposed to do or it doesn’t. It’s this very property that makes things like ralph loops possible, as you can give the LLM a set of success criteria and then free it to iterate toward a solution.
Second, and this is where open source plays a deeply ironic role in this business: enormous quantities of source code is widely and publicly available for use in LLM training. There is truly no other industry that has a training corpus as rich and varied as software development.
Third, the AI industry is itself made up of software professionals, and as a result they are uniquely positioned to create incredibly effective tools for generating code with AI. In fact all major AI labs end up dogfooding their own tools to create those tools, which only speeds up the rate of iteration and advancement.
The end result: the tech industry, from the C-suite to the coding grunts on the ground, is shocked and terrified about what’s to come as, for the first time, the call is coming from inside the house.
I’m honestly not sure how I feel about this whole situation.
My first instinct is to laugh and shake my head.
One need not look very far to find indignant software developers absolutely certain that their jobs cannot possibly be automated away by the very tools their industry contemporaries are creating to replace them. I suspect you’d also not have to look far into their posting histories to find those same people comparing cabbies to buggy whip makers.
Meanwhile, on the other end of the spectrum, you have CEOs and CTOs pushing folks to token maxx while pounding the table that we all have to AI harder and faster lest we be tossed into the dustbin of history.
The result is a bizarre combination of denial and boosterism that’s hard to square.
Ultimately, if we’re to take any lessons from the very industries tech has disrupted, one has to assume a few things:
First, we’ll see a race to the bottom on price. We can clearly see this trend among writers, musicians, the ride sharing industry, food delivery, and even online app stores. On the flipside, we’re likely to see a cambrian explosion of content (see: Dana Lawson predicting “a billion new apps”) at rock bottom prices, though as with apps, books, and so forth, the signal to noise ratio is likely to be so low that it’ll be impossible to discern the quality products from the slop.
Second, workers will be squeezed as wages are driven down and the availability of AI diminishes the perceived value of expertise. We’re already seeing this trend among various commercial artistic professions, including graphic designers and copywriters.
Third, once destroyed those industries will not meaningfully return. This is not a process that will result in a natural ebb and flow. Once the software development profession is disrupted, it will stay that way as talent pipelines dry up and expertise ages out.
Fourth, we will not find ourselves working less. Rather, there will be a bimodal outcome: those unable to find a place in the new AI-powered industry will fall out of it entirely, while those that remain are worked harder and harder as they drive automated development systems. Five PRs a week? Hah! Try fifty. Or five hundred.
Fifth, and finally, wealth will, as it always does during times of mass automation, be captured by business owners, senior executives, and the investor class, as well as model providers who will act as rent seekers. You think the wealth gap is bad now? It’s only just begun.
These tools are here. They’re not going away. The question, then, is: now what?
The only real potential counterweight to the disruptive effects of AI is a combination of government taxation and regulation, in order to redistribute wealth and put in place programs that can help society move through this transition, and organized labour that can act as a bulwark against corporate excess.
But given the perennially libertarian bent of the tech industry, I see neither of these things playing out.
Instead, we’ll all engage in creative destruction together, each hoping and believing deep down that, yeah, it might come for that other guy, but surely it won’t come for me.
Until it does.
Because the cannibals are hungry. And to them we’re all just meat.
As a bit of a postscript, you might wonder how this piece squares with this one, which ends on what one might call an optimistic tone.
And the reality is: I think they’re both true. As I noted in the introduction to that previous piece:
The more I work with these tools–and I work with them quite literally every single day–the more I realize just how utterly disruptive they are to the industry. I cannot be honest and deny their incredible power and utility.
I also cannot deny that this technology threatens lives and livelihoods, accelerates climate change, drives wider the already catastrophic wealth gap, distorts political systems, and drowns out human voices in a sea of AI slop.
Ultimately, as an individual creator, AI is enormously powerful, and I’m happy to put it to use to advance causes that I care about, including the indieweb and small web.
But as with any tool, it comes down to those who wield it and those who decide where and when to use it. And given the kinds of leaders I’ve seen in the industry, well, let’s just say they’ve not earned my confidence.