For years, the argument over artificial intelligence has followed a familiar script. Technology companies build increasingly powerful systems, governments struggle to keep up, and Silicon Valley warns that politicians risk regulating a technology they do not understand. The disagreement is usually presented as a question of speed: how much oversight can AI tolerate without slowing innovation, weakening American competitiveness or handing an advantage to China?

Dean Ball is raising a more fundamental question.

Ball, who recently joined OpenAI as Head of Strategic Futures, has argued that frontier AI labs could become a “counterbalance to government.” Before joining the company, he described the organizations building the most advanced AI systems as a “new kind of institution under the sun.” Taken separately, those phrases might sound like the kind of ambitious language that surrounds almost everything in artificial intelligence. Taken together, they point to a much more consequential idea: that companies such as OpenAI may eventually become powerful enough that it no longer makes sense to think of them simply as corporations operating within rules set by the state.

That possibility is worth taking seriously, not because OpenAI has formally declared itself a rival to government — it has not — but because the technology industry is beginning to talk openly about a future in which the old hierarchy between governments and companies may no longer be so clear.

If the people building frontier AI are right about where this technology is heading, the implications go well beyond chatbots and productivity software. Advanced models could become deeply involved in software development, scientific research, medicine, education, cybersecurity, finance, military analysis and the operation of companies themselves. Governments will use these systems too. Intelligence agencies will use them. Public administrations will use them. Critical infrastructure operators will use them. At some point, a government may find itself trying to regulate an AI provider whose technology it simultaneously depends on.

That is not a normal relationship between a company and a regulator.

When a Technology Company Starts Looking Like an Institution

We have seen hints of this before. Governments already rely heavily on privately controlled cloud infrastructure. Apple can decide which software reaches more than a billion devices. Cloudflare can determine whether parts of the internet remain accessible under attack. Google has enormous influence over what information people find. Microsoft and Amazon operate infrastructure used by governments, militaries and critical industries around the world. None of those companies is sovereign in the constitutional sense, but each controls something that modern states cannot easily reproduce.

Artificial intelligence could take that dependency much further because AI is not infrastructure for a single narrow purpose. It is increasingly being developed as infrastructure for cognitive work itself.

If one system can write software, identify cyber vulnerabilities, analyze intelligence, design molecules, interpret legislation, generate political messaging and run business processes, then control over that system is a particularly unusual form of power. The company operating it may be able to influence entire categories of economic activity simply by changing prices, access rules, safety policies or technical capabilities. Decisions that currently look like product management could begin to resemble public policy.

This is what makes Ball’s idea interesting. There is, in fact, a respectable argument for institutions capable of counterbalancing governments. Democratic societies are deliberately full of them. Courts restrain elected politicians. Independent media expose government misconduct. Universities challenge political orthodoxy. Companies can reject unlawful demands. Civil society organizations exist precisely because concentrated state power is dangerous.

It would therefore be a mistake to assume that governments should automatically control the most capable AI systems simply because they are governments. States have their own record of surveillance, censorship, political coercion and technological abuse. A government with exclusive control of extremely powerful AI could be considerably more frightening than an AI industry with multiple independent actors.

Ball’s position is also more nuanced than a simple demand for deregulation. He has argued that certain future AI capabilities may be so dangerous that they “cannot realistically be left in private hands.” Systems capable of enabling biological weapons, highly advanced cyber operations or other forms of catastrophic harm could eventually require much stronger state involvement. In other words, the argument is not necessarily that private companies should replace governments. It is that the boundary between private and state power may have to be redrawn.

But once AI companies begin thinking of themselves as institutions capable of balancing governments, an obvious problem appears: governments have counterweights too. Who provides the counterweight to the AI companies?

Power Without Elections

That question is harder than it sounds because corporations were never designed to carry this kind of institutional legitimacy. Governments, for all their failures, are built around mechanisms intended to constrain power. Elected officials can be removed. Government decisions can be challenged in court. Public institutions operate under administrative law, constitutional limits, disclosure requirements and political scrutiny. These protections are imperfect and vary enormously between countries, but the principle is clear: the more coercive power an institution possesses, the more society expects mechanisms that make that power accountable.

The chief executive of an AI company is accountable in a very different way. Users do not vote for the board. Model policies are not passed by parliament. Internal safety decisions are generally not subject to freedom-of-information laws. A company can reorganize its leadership, rewrite its terms of service, discontinue a product or alter access to a model without anything resembling democratic due process.

For ordinary software, this is mostly a commercial matter. If an email application changes its terms, customers can complain or leave. But imagine an AI system that has become embedded in hospitals, universities, government departments and thousands of companies. Imagine that millions of workers rely on it to perform their jobs. Imagine that government agencies depend on it for cybersecurity or analysis.

At that point, the provider starts to look less like a software vendor and more like critical infrastructure.

And societies treat critical infrastructure differently for a reason. Banks face capital requirements because their collapse can damage everyone around them. Utilities have continuity obligations because electricity cannot simply disappear when supplying it becomes inconvenient. Telecommunications companies operate under rules that reflect their importance to society. The more indispensable a service becomes, the less comfortable we are leaving every decision about it entirely to the discretion of its owner.

If frontier AI genuinely becomes as important as its developers expect, similar questions will eventually arrive at the doors of AI companies. Should providers of economically critical models have continuity requirements? Should customers have some form of appeal if access is terminated? Should governments be able to require interoperability or emergency access under narrowly defined circumstances? Should there be independent auditing of systems used inside critical infrastructure? What happens if a model provider decides to withdraw from a country on which hospitals, businesses and public agencies have become dependent?

None of these questions has an easy answer, and rushing to regulate hypothetical problems can create problems of its own. But they illustrate why describing AI companies as a counterweight to government has consequences. Institutional power rarely comes without institutional obligations.

If AI Replaces Work, Who Replaces the Tax Base?

Taxation is one of those obligations, although probably not the most important one.

If AI systems automate a substantial amount of human work, governments may eventually face a practical fiscal problem. Modern tax systems collect enormous amounts of revenue from employment. Workers pay income taxes. Employers contribute payroll taxes. Wages fund consumption, which generates additional tax revenue. If some of that work moves from salaried employees to machine intelligence, the economic output may remain — or even grow dramatically — while the way governments collect revenue changes.

This does not automatically justify a punitive tax on artificial intelligence. Taxing emerging technology simply to make automation less attractive would be a good way to protect inefficient incumbents and push innovation elsewhere. But if AI companies are serious when they predict systems capable of replacing meaningful amounts of cognitive labor, policymakers are justified in asking whether the tax system should eventually follow the activity.

One proposal is a so-called token tax: a very small levy on large-scale commercial AI inference. Tokens are attractive because AI providers already meter them for billing, meaning usage is observable in a way that the work of a vaguely defined “robot” is not. Yet tokens are also an imperfect tax base. Different models tokenize language differently, reasoning systems may use large numbers of internal tokens, self-hosted models complicate enforcement, and tomorrow’s AI architectures may not use tokens in anything resembling today’s form. Compute, energy use, corporate profits or some other measure could prove more sensible.

The precise mechanism matters less at this stage than the principle behind the discussion. If increasingly large amounts of economic activity move from human labor toward privately controlled machine intelligence, governments will eventually have to decide whether existing systems for funding public services still make sense.

And taxation is only one example of a broader issue. Governments do things besides collect money. They provide continuity, security, dispute resolution, emergency response, infrastructure and, at least in democracies, a mechanism through which the public can challenge the exercise of institutional power. If AI companies want to be regarded as something more consequential than ordinary businesses — institutions capable of balancing states — the conversation cannot stop at their freedom from government control. It also has to include the responsibilities that come with becoming indispensable.

OpenAI Is Already Living the Contradiction

There is a particularly interesting tension here because Ball’s views should not be mistaken for an official declaration of OpenAI policy. OpenAI itself has advocated a durable federal framework for frontier AI and has spent considerable effort working with governments on safety, infrastructure and national security. The company is not publicly arguing that the state should disappear from AI governance.

If anything, the coexistence of those positions tells us something important about where the industry is heading. AI companies may want government involvement in some areas while resisting it in others. Governments may attempt to regulate AI companies while simultaneously becoming dependent on their technology. Both sides may possess leverage the other cannot easily ignore.

The relationship could eventually resemble something closer to negotiation between powerful institutions than traditional regulation of an industry.

That prospect should make people on both sides of the political spectrum slightly uncomfortable.

Those suspicious of government power should consider how much authority they are willing to place in corporations that have no democratic mandate. Those who favor strong regulation should consider whether concentrating advanced AI capabilities inside the state creates dangers of its own. And technology companies should recognize that the argument works both ways: if they want society to accept that frontier labs deserve independence because they can serve as a check on government, society is entitled to ask what checks should exist on them.

The Real Question Is Not Who Wins

Perhaps this entire discussion will turn out to be premature. AI progress may slow. Models may become commoditized. Open-source systems may prevent any small group of companies from accumulating the kind of power now being imagined.

But the people building frontier AI clearly do not believe that is the most likely future. They are spending hundreds of billions of dollars on infrastructure, recruiting policymakers and national-security officials, and openly discussing systems that could transform large parts of the economy.

We should probably take their ambitions seriously enough to think through what follows from them. The most consequential question raised by the idea of AI labs as a counterweight to government is not whether OpenAI, Anthropic or Google should be allowed to become powerful. They probably will become more powerful if their technology continues to improve.

The real question is what happens when a private company becomes powerful enough that “it’s just a company” is no longer an adequate description of what it is.

Post Views: 6