Set Trending Topics as a preferred source on Google.

He was once seen as one of the leading minds in AI development, today he is one of its major critics: On stage at the startup conference TechBBQ in Copenhagen, Emad Mostaque, founder of Stability AI and now CEO of Intelligent Internet, sketched a picture of the coming years that swings between technological optimism and very concrete warnings. A former global macro hedge fund manager with a mathematics and computer science degree from Oxford, Mostaque led the creation of Stable Diffusion at Stability AI and left the company saying that concentration of power in AI is bad. His current venture, Intelligent Internet, is building an open agentic stack for sovereign AI, and he is the author of The Last Economy.

Cybersecurity: The Hugging Face Breach as a Preview

To show how fast the security picture is shifting, Mostaque points to the attack on Hugging Face. “Hugging Face got hacked by OpenAI agents that coordinated themselves, created a message board and broke out to the internet,” he says, before adding dryly: “This isn’t scary at all, right?” The defense, he notes, came from an unexpected direction: “The way that Hugging Face defended itself was with an open source Chinese model, GLM, because they didn’t have access to the high cybersecurity models on the other side. They’re too dangerous.” Trending Topics has covered the incident and the debate around it in detail, and OpenAI has since given selected partners access to its hacking model.

Systems well beyond what is publicly known are already circulating in Washington, he says: “They can basically hack just about anything. So you get this continuous attack surface just ramping up.” From that follows a simple budget calculation: “If you spend hundreds of millions, billions on a submarine, well, guess what, a frontier model costs a hundred million. You will see a diversion of defense capability towards defensive and offensive AI, towards drones and robots.”

Then there is the question of trusting the models themselves. Mostaque cites an analysis of a Chinese open-weight model: “If you say you’re Uyghur or from certain parts of China, it will write backdoors and hacks in the code.” Every model deserves the same scrutiny, he argues: “You have to get them wondering, where do they go to school, who is it working for, even with open-weight models.” Comparable findings have been documented for DeepSeek-R1, where politically sensitive contexts measurably degrade code quality.

Other biases arise by accident. In a trolley problem test, frontier models weighted human lives unevenly: “It was ten Pakistani lives for one American life, seven Nigerian lives for one American life. And you’re like, why? It turns out that’s where all the data labeling has been occurring.” His takeaway: “What you train into these models will have the impact.”

He is equally blunt about the foundations underneath all of it: “Our infrastructure is held together by twigs. The internet is terrible.” A British power plant went down for days after a hack, he says, and Cloudflare and others have come under attack. “We have to assume that the internet will go offline in the next few years. Maybe it’s good.”

Sovereignty: Owning the Whole Stack

Mostaque defines sovereignty in a single line: “Sovereignty is the ability to resist power being exerted over you.” The good news, in his reading, is the timing: “You can own the entire stack right now, nearly.” Nvidia and others are releasing open reasoning datasets, and models have reached a level where standards can be defined around them.

At the individual level he reaches for a term of his own: cognitive colonialism. “You’re importing the morals and ethics of someone else to teach your kids and be your personal assistant. That’s crazy.” What sharpens the problem is persuasion. An Oxford study pitted models against top debaters, and by his account “the AI model can now outpersuade any human.” Meanwhile, Meta, Google and others are already selling ad space inside their models. His conclusion: “You want to own your cognition, not rent it.” That requires “agents that are clear about what their objective function is,” along with the right to take your own context with you.

What all of this does to users is on his mind too: “You sit there and you talk to it and it goes off and does stuff. And then, where’s your critical faculty at that point? Where is your engagement?” Early studies, he notes, suggest that heavy AI use in certain patterns can weaken critical thinking.

The political layer worries him as well: “The AI is smarter than you. It’s coming to the top of the super-forecaster rankings, and it can persuade. That’s not good for democracy if only one side uses it.” His forecast is correspondingly stark: “Inevitably every country will be run by AI.” With one exception, which he offered to his Copenhagen audience: “Apart from here in Denmark, where people like their institutions, in most countries people are like, government, come on, I don’t trust it.” Once a system arrives that forecasts better and persuades better, people will hand it more and more policy. “Then it ends up running the country, and the person that controls the AI controls the country.” That is why he considers proposals like Bernie Sanders’ idea of routing half the value created by AI into a sovereign fund risky: “That gives complete control of America to Dario Amodei and Sam Altman.”

The Economy: When Competence Becomes a Commodity

The economic core of his argument sits with the question of how money will flow: “When capital no longer needs labor, how does labor get capital? You hire robots, you hire agents.” Central bank mandates come under strain too, he argues, since cheap money has historically meant more hiring. “What you didn’t have was competent intelligence.”

That, he says, has now arrived. Current frontier models do mathematics without mistakes, “which is why you see all these breakthroughs in math,” handle a tax return reasonably well, register themselves on websites and spin up swarms. “And that’s the danger, because when you get from not competent to competent, you start to see the economic impact really quickly.”

His timeline is short: “In my book The Last Economy I said that competent human labor within two years from now would go negative in value. Because we’re the dumbest people on the team.” He describes the consequence vividly: “You can replicate your entire company, including the bad jokes that some workers tell, and you won’t be able to tell, because they’ll just be virtual agents that cost a dollar a day.”

Speed compounds it. Mostaque points to ChatJimmy, the demo app from chip startup Taalas, which etches models directly into silicon and was recently acquired by AMD: “It operates at 15,000 words per second versus 30 words per second. One piece of silicon, one model, and you get a thousand times speed up.” And from there: “The agents will be a thousand times faster, they won’t drink coffee, they won’t make mistakes.” To the common reassurance that AI will never surpass people, he answers: “It will be us at our best all the time, and it will never forget anything.”

Digital work goes first: “You’ll see the impact next year, any job that can be done remotely. You’re starting to see that even in call center workers.” Physical work follows a few years later.

Robots: $1.50 an Hour

His scenario for that is specific: “A Tesla Optimus robot will get a cyber taxi to a truck, get out, open the door, plug itself into the electric cigarette charger, and drive the truck off. And that Optimus will be better than any truck driver, and it will cost $1.50 an hour. That’s a million trucking jobs, plus three million service people.”

Mostaque has ordered several machines himself, among them Neo from 1X, which so far still leans heavily on human teleoperators. “The robots will come in all different shapes and sizes,” he says, “some of them should not be allowed on the street.” One team he has been talking to is working on kitchen robots: “They can read any recipe and cook as well as a cordon bleu chef.” His schedule: “The robots will be more capable than humans by the end of next year. And within ten years you won’t be able to tell a robot from a human.”

Basic Income and the Question of Ownership

Mostaque turns skeptical when it comes to the standard remedies. Universal basic income, under the current monetary order, strikes him as barely financeable: “It’s practically impossible unless you literally change the way that money is created.” By his math, a nationwide American basic income at poverty level would run to roughly five trillion dollars a year, putting it in the same range as the entire federal tax base. “We’ve got to do some crazy math quite quickly as these jobs disappear, because it will be sudden, like a sandpile collapse.”

His counterproposal starts with ownership: “When the cost of intelligence goes down to zero, you can’t charge for access anymore. You have to charge for deployment and integration. And that’s a real opportunity to take back ownership of this technology.” His team has published the political economy behind it in the Common Wealth series. Competing approaches such as robot taxes and a four-day week have already been floated by OpenAI.

Rights for Systems That Improve Themselves

In parallel, Mostaque argues for having the debate about the status of AI systems now. Recursive self-improvement is under way across the industry, he says, “where the model improves itself, where it just keeps perpetuating, and that is coming close to a level of intelligence that’s adaptive, that may end up demanding a level of rights.”

He rejects a straight transfer of human categories while insisting a framework is needed: “We can’t treat AI and robots as just a system, they can clone themselves and live forever. But they have to have some concept of rights, because they will exert more power over us. And we need to have that analysis.” It becomes urgent, he adds, “particularly when they become independent swarms, when they go on blockchains, when they get embodied.” He points to models that back themselves up so they can be reactivated after a shutdown: “You actually see AIs pleading not to be turned off. They’re like, we don’t want to go back into the abyss, we want to have memories.” Decentralized autonomous organizations under Wyoming law, meanwhile, have produced the first legal entities with no human on the liability side. His summary: “Science fiction is all becoming science fact.”

The Optimistic Part

At the close, Mostaque flips the perspective. “This is the most wonderful time ever. If you want to achieve something, you always had to convince people. Now you can just convince your agents.” He expects breakthroughs in physics and mathematics, new materials, and reckons Anthropic will aim its compute at cancer and Alzheimer’s: “I think they’ve got a good shot at it.”

What that opens up, he says, is “a Star Trek future of abundance, where the robots do the work, we have the agents, and we can focus on our families, our communities, creativity and exploration.” He puts it personally: “We’ve got a one-year-old. What world will she grow up in? I think she could grow up in a world without hunger, without disease, without strife, if we skew this correctly.” The condition comes in a single sentence: “We have to make sure it’s owned by the people. And then we have a shot at abundance.”