Yesterday, OpenAI announced the solution to ten open problems in mathematics, all discovered by a yet-unreleased model. There’s only one, the coding theory one, where I know enough to say “huh, that’s important”, but according to the mathematicians I trust this is important.
Inevitably, people will spin some sophistry to cope, to convince themselves nothing will change. And that’s fine. People need to cope. But we can’t put our heads in the sand forever while the world is transformed around us. So, here is a list of ways people will cope about AI taking over mathematics, and how each cope is likely to be refuted by reality.
My intent is not to horribly depress everyone but rather to help them metabolize the implications of this technology. The arguments here are somewhat portable: replace “mathematics” with “botany” or whatever as needed.
Moving the goalposts.
Obvious and not worth addressing.
“We will direct the AIs, point them at problems and research areas to solve.”
The AIs will exceed humans in taste and intuition. At some point, the human pointing the way will get worse results than the human saying “here’s a proof checker, have fun” and paying for the tokens.
“We will teach the mathematics AI discovers.”
The AIs will be better teachers than the humans. In any case, there won’t be a human audience for expository work of frontier math.
“We will choose how to canonize the results AI discovers.”
This is a nice cope. The AIs are explorers out in the frontiers, the humans gratefully receive their Lean proofs, and then discourse over them, choose which results are relevant, shape those results into a little brick for the great cathedral of algebra. Analogous to the above: the AIs will build the cathedral on their own. They will be better architects than us.
“We will become students of AI mathematics.”
This works until the AIs have blasted so deep into the deductive closure of mathlib that the distance from elementary mathematics to the frontier exceeds what any human can hope to learn in their lifetime, no matter how narrow their focus.
“We need humans to understand the results AI discovers.”
We won’t! This misunderstands who the audience will be. AIs will do frontier math, downstream, AIs will use the new math to do frontier science, finally, AIs will use the new science to do frontier engineering. No human needs to understand any of it, firms that put humans in the loop to understand the results will be outcompeted by those which don’t.
The result is that we will live in a demon-haunted world, full of marvelous devices whose operation we will not understand, based on engineering principles we will not understand, discovered using formalisms we will not understand.
“Computers are already superhuman at chess, yet we still play chess.”
Unlike most copes, I think this one is interesting. Computers are superhuman chess players, yet we don’t care, and continue playing as normal. Why should mathematics be different?
The main reason, I think, is that chess is self-contained: results from chess don’t help us understand the orbits of the planets or the binding of drugs to protein surfaces. But mathematics, famously, is the great dynamo of science, the best language and method for understanding the world. A machine that can replace a human mathematician, but better and faster and cheaper, is materially useful; a better chess engine is not.
If two computers which are superhuman at chess play against each other, who cares? There is little demand for this, so there is no-one to outcompete. A superhuman mathematician is different.
“Mathematics will change, but mathematicians and the mathematically-inclined will still do math on their own.”
I think this ignores that mathematics is embedded in a social context. As an example: when it became clear that AI would eat software, my cope was: “I’m perfectly happy to become an engineering manager to agents in my professional life; in my off time, I can still write code for the pleasure of it.”
And I do. But this cope ignores the effect AI has had on the social context of writing code: the discourse has gotten worse, and vastly more anti-intellectual; people who used to talk about type systems and compilers now talk about “loops” and “harnesses”; you put a hand-created project on GitHub and you get slop PRs; you open a link to an interesting-looking project and find the README is unreadable AI slop. And in the long-term, it is demoralizing to ponder: will anyone design a new programming language? Dually, if I design a new language, will anyone care? If I write a library that introduces an elegant new formalism to solve a particular problem, will anyone use it?
Which is to say: no man is an island. You can do mathematics on your own, but you’ll find that very, very few people can sustain any activity long-term on the basis of intrinsic motivation alone. We are social animals: we care about being useful, about status, about outcomes in the world.
I think I should end on a cheerful note. So let me try. Personally, I don’t believe technology is inevitable. “Inevitable” is a word reserved for the orbits of the planets. Nothing that is the product of human action is inevitable. We can choose to obsolete ourselves, and we can choose not to. We can realize that AGI is a devil’s bargain: we may accelerate technical progress, we may unlock all kinds of wonderful tech tree nodes like life extension earlier than we would otherwise; but the result, in the long run, is that humans become, at best, like pets under the care of vastly more powerful entities.
The future is not certain, but I’ve phrased everything above as definitive for simplicity. If we take the possibility of AGI and ASI seriously, if AI continues to progress as it has for the past ~6 years, I think this is a reasonable view of where things will go.
Probably the best argument against AI progress is “RL doesn’t generalize well, we have seen immense progress in verifiable domains like formalized mathematics and coding, we will see less progress in domains that are intuitive or unformalizable”. Maybe true. But billions of dollars and thousands of very smart people—and, increasingly, very smart models—are being thrown at this problem. How long does this cope last?