Can AI do maths? Not long ago the question sounded like a joke. Early ChatGPT fumbled long multiplication, and in July 2024 people were still sharing examples of it insisting that 9.11 was bigger than 9.9. In 2026 the headlines turned round. In May an OpenAI model disproved a conjecture Paul Erdős posed in 1946, and in early October OpenAI released more than 700 mathematical manuscripts from a model nobody outside the company can use, including a claimed proof of the Unique Games Conjecture that specialists are still trying to read. So the question people ask now is sharper than it used to be. Can the chatbot on your screen be trusted with a number that goes into a client report? And if machines are settling problems that mathematicians left open for decades, is general intelligence just around the corner? Anyone who uses these tools for work has a stake in the first question, and the second is a big part of the wider argument about why maths progress doesn’t make ChatGPT AGI.
I come at this as someone who studied maths without ever loving it. I took A-level maths, physics and computer science and went on to a First class degree in software engineering, yet pure maths never came easily to me, and I won’t pretend to judge a research proof myself. In ChatGPT’s early days I asked it to predict Bitcoin’s price, and a Reddit commenter told me that’s not how LLMs work: they are pattern-recognition systems, “not wired to do maths”. This autumn at Opace, the digital agency I run, that old claim came back to me twice. The first time was during a 25-day study, from 16 September to 10 October 2026, in which we asked AI to come up with a genuinely new, patentable invention (the full account of why AI couldn’t invent anything patentable has its own article), and which meant a great many sums about forces, energy and probabilities that we could check afterwards. The second was on the study’s last day, when I tested ChatGPT directly, spending 46 minutes questioning it about its own maths ability and pushing back on its answers. Neither was a page of exam sums, since published benchmarks already do that job with thousands of problems, so what our tests add is a view of how AI maths holds up in real work and under questioning, which we’ve set against the published research throughout.
Can AI do maths? The short answer
So, can AI do maths? Yes, from arithmetic to research-level proofs, though the answer depends on which AI and which kind of maths: chatbots such as ChatGPT reason well and calculate reliably when they run code, specialist systems with proof checkers have settled problems that were open for decades, and the trouble starts wherever nobody checks the answer, which includes forecasts and attempts at invention.
Our own testing, described in full later, found the following:
- Asked to rate its own maths, ChatGPT said it understands concepts and can reason through problems, though not infallibly, but can’t guarantee an exact calculation by itself, and the published research agrees with that split.
- Every example of AI mathematical discovery it gave me depended on something outside the language model to check the answer: an automatic evaluator, a proof checker or human graders.
- In 25 days of AI working on real product ideas, starting with a power-assisted shopping trolley, the models’ unchecked numbers were sometimes badly wrong. ChatGPT said shoes that harvest energy from walking produce a few watts, when the research puts them at 8 to 58 thousandths of a watt. Asked how hard you’d need to push a loaded trolley up a kerb, different AI agents gave answers ten times apart for similar cases.
- Each of those errors was caught by a second model or by recalculating with worst-case values, never by the model that made it.
- The study produced 276 distinct ideas, each checked against existing patents and products, and none was strong enough to justify a patent application, in large part because invention has nothing like the automatic checks that maths enjoys.
- When I pushed back, ChatGPT gave ground four times in one conversation, and it accepted a claim I’d picked up online, that AI maths ability was growing tenfold a month, without asking where it came from.
Before getting to those findings, it’s worth being clear which machine is doing the maths, because the word “AI” covers several very different things here.
Which “AI”? Chatbots, code and proof machines
When people ask whether AI can do maths, they usually mean a chatbot (ChatGPT, Claude or Gemini answering in a text box), which is only one kind of AI among many. Computer algebra systems have done exact symbolic maths for decades without anyone calling them intelligent, and AlphaFold predicts protein structures without doing anything a maths teacher would recognise as maths. We’re mostly answering the question for large language models (LLMs), and we’ll point out where purpose-built systems change the answer, because in maths they change it a lot.
Even inside the LLM family, three quite different machines get lumped together, and most confusion about “AI maths” comes from mixing them up.
The first is a chat model working on its own. You type a question, and the model writes an answer token by token from what it learned in training. It can lay out a method, explain why the method works and often get the right number, but nothing outside the model checks the digits. When ChatGPT explained to me why its early versions struggled, it said they generated answers from patterns learned in training instead of passing the sum to a dedicated arithmetic processor. That is still the right description of a chat model answering without tools today, even though the patterns have become far better.
The second is a chat model that writes and runs code. The model still decides the method, but it hands the arithmetic to Python, which does it exactly. ChatGPT drew this distinction itself: without tools it reasons through the problem inside the neural network, and with Python it writes instructions for a separate program to carry out. That only protects you if the code is right and actually ran, which a chat window doesn’t always make obvious.
The third is a language model inside a loop with a proof checker. The usual checker is Lean, a programming language and proof assistant in which every step of a proof has to compile, much as code has to compile, before the proof is accepted. Systems built this way generate a candidate, let Lean reject it, feed the error back and try again, often thousands of times.
Why does this matter so much for the question? Because the evidence people throw at each other comes from different machines. Someone who watched a chatbot fumble a long multiplication in 2023 is describing the first. Someone citing an Olympiad gold or a disproved conjecture is describing either the third or an advanced reasoning model given far more thinking time than the public app offers, sometimes one the public can’t use at all, and both of them can be telling the truth about what they saw.
One more distinction is worth keeping in mind as you read, between a generated response, which is text the model believes is right, a tool result, which is a number some other program computed, and a checked proof, an argument a formal system has accepted step by step. They carry very different weights, yet an answer in a chat window can contain all three without labelling which is which (and I’ve mixed them up in my own thinking as often as anyone).
Five levels of doing maths
“Doing maths” covers jobs that a teacher would never mark the same way, and most arguments about whether AI is good at maths are really arguments about which job is meant. We find it easiest to think of a ladder with five rungs:
- Arithmetic: getting exact digits out of a sum.
- Methods: algebra, trigonometry, calculus and statistics done by known procedures.
- Competition problems: unfamiliar puzzles with known answers, such as AIME or the International Mathematical Olympiad (IMO).
- Proof: an argument that something is true in every case, not just the ones you tried.
- Open problems: questions nobody has answered, plus the harder job of choosing which questions are worth answering at all.
Each rung has dated evidence, and the pattern is odd: chat models were weakest for longest on the bottom rung, where a pocket calculator wins, they now look strongest on the middle rungs, and the top rung is where the most dramatic headlines sit and the hit rates are lowest. The sections after this one take that evidence in date order, from GPT-3’s arithmetic in 2020 to the open-problem results of 2026.
Can AI do calculus?
Calculus shows why the ladder helps, because differentiating a product of functions or integrating by parts is a procedure, and procedures are what models learn well from millions of worked examples. ChatGPT’s own list of what it can tackle puts calculus alongside statistics, probability and linear algebra. The benchmark evidence is a little indirect. The MATH problem set behind many headline scores stops at precalculus, but when OpenAI described its o1 reasoning model in the research post Learning to reason with LLMs in September 2024, it also charted exam results, and on AP Calculus o1’s raw score was 83.3% against 71.3% for GPT-4o. In our experience a modern chat model will usually get standard calculus right and explain it well. Where it slips, it tends to be in the algebra around the calculus, and the FAQ at the end has two worked examples you can check by hand, with the check we use ourselves.
Can AI prove theorems?
There are two quite different things a model can do here. A chat model can write out a known proof, such as the irrationality of √2, because thousands of versions sit in its training data. Systems with a checker can do something stronger and produce proofs that Lean accepts line by line, including proofs of statements nobody had proved before, and several of the 2026 results on open problems later in this article are of that second kind.
What is the difference between finding a number and proving a theorem?
Kevin Buzzard, the Imperial College number theorist leading a Lean formalisation of Fermat’s Last Theorem, made the distinction clearly in a 2023 talk, “Mathematics and the computer”. A calculation is something like computing the first million prime numbers. Reasoning is proving that there are infinitely many. He noted that tools like ChatGPT can reproduce well-known proofs because they have seen so many of them, and that hoping a computer will automatically prove the Riemann Hypothesis “is still science fiction”.
That split runs through every benchmark. Many tests, including the original FrontierMath and AIME, ask for a final number that a script can mark automatically. A proof can’t be marked that way unless it is written in a formal language. So when you read that a model “solved” a problem, it helps to ask whether it found an answer that could be checked against a key, or proved something that a checker or a mathematician accepted, and we’ve leaned on that question more than on any other line of the ladder while writing this.
Why is AI bad at maths? What tripped up early ChatGPT
A language model was never built to be a calculator, because it is trained to predict the next piece of text, and for a sum that means predicting the next digit of an answer it hasn’t worked out the way a calculator would. ChatGPT’s own explanation of its early weakness lists the failure modes I remember from 2022 and 2023: reproducing familiar equations without executing them, confusing similar-looking numbers, slipping when carrying digits, and producing a convincing explanation for an answer that was completely wrong. That last one did the most damage to trust, because the wrong answer arrived in exactly the same confident tone as the right one.
Size was the first obstacle, since every extra digit is another chance to slip. The clearest published picture of this is still the GPT-3 paper from 2020, “Language Models are Few-Shot Learners”. Its authors gave GPT-3 2,000 random sums per task, with a few worked examples in the prompt. It got two-digit addition right every time, but accuracy fell to 80.4% at three digits, 25.5% at four and 9.3% at five. The same tables show how much the wording of the request mattered. On three-digit addition the largest GPT-3 model scored 34.2% with no worked example, 65.5% with one and 80.4% with a few, and on two-digit addition the same three settings gave 76.9%, 99.6% and 100%, so the sums didn’t change, only the prompt, which suggests the model was pattern-matching its way towards a method.
The same paper found that the next model down, at 13 billion parameters, could manage two-digit addition and subtraction only about half the time. Arithmetic appeared with scale, but it appeared unevenly, and nothing in the chat window warned you which side of the cliff your question sat on. GPT-3 was also the model we first wrote about at Opace, in December 2022, when we saw it mainly as a GPT-3 tool for SEO text generation, and our January 2023 explainer on what GPT-3 was before ChatGPT covers how it differed from the chatbot most people met.
“Can AI do math?” Why the old answer was “no”
The commenter who told me LLMs weren’t built for maths had plenty on their side in 2022. ChatGPT fumbled simple calculations often enough in those days that I could hardly argue, and the consensus among people who understood the technology was that a system trained to predict text would never be dependable with numbers.
The hype of 2023 pulled the other way. In November that year Reuters reported, from unnamed sources, that an internal OpenAI project called Q* could solve certain maths problems, though only at the level of grade-school students, and that some inside the company saw it as progress towards general intelligence; our own history of OpenAI and the Q* story covers that episode. When I raised Q* with ChatGPT, it pointed out that Q* was never documented publicly in enough detail to say how it worked or whether it led to later models, which was a more cautious answer than much of the coverage at the time.
Looking back on the early criticism in October 2026, ChatGPT split it in two. It accepted that the criticism was partly justified at the time, but said the general claim, that pattern-recognition systems can’t learn maths, had always been too simplistic. Its illustration was a child who learns that 7 × 8 = 56 by memory before learning a method that works on sums they have never seen: learning patterns and learning methods can happen in the same system. That is a reasonable answer, and it fits the GPT-3 chart above. A pattern-trained system had already picked up a working method for small sums in 2020, long before anyone bolted on a calculator.
Why “strawberry” and 9.11 became 2024 party tricks
Part of the reason sums went wrong lies in the way digits reach a model, which doesn’t see 847,392 as six digits. OpenAI’s open-source tokeniser for GPT-4o, tiktoken, splits it into 847, a comma and 392, and it splits the answer to 847,392 × 63,719, which is 53,994,970,848, into chunks of three digits that don’t line up with how you would carry a one, so the model has to learn arithmetic over those chunks.
That chunking explains two famous 2024 failures. Asking a chatbot how many r’s are in “strawberry” became a party trick because, in a question like that, GPT-4o’s tokeniser passes the whole word as a single token, so there are no letters inside it for the model to count. The decimal version was even more striking. A user on OpenAI’s own community forum reported on 17 July 2024 that ChatGPT insisted 9.11 was bigger than 9.9, and others replied with the same result.
A reply in the same thread on 20 August 2024 said the problem looked fixed in GPT-4o, so the failure belongs to a particular moment in 2024, when a model could still trip over something a ten-year-old gets right.
There was a deeper problem too, and the cleanest evidence of it came in March 2025, when Anthropic published its research on tracing the thoughts of a large language model. Its researchers traced how Claude adds 36 and 59. Inside the model, one path estimated the rough size of the answer while another worked out the last digit exactly, and the two combined to give 95. When asked how it got there, Claude described the schoolbook method of carrying the 1, which isn’t what its internals did. We find that uncomfortable as people who use these tools for maths, because it means a model’s account of its own working is a plausible story that you can’t safely read as a log of what it actually calculated.
Simple sums could still go wrong in 2025, though far more rarely. In Opace’s guide to what GPT-5 is, Adam Bullas, one of our writers, described GPT-4o getting a simple addition wrong that year when he used it in place of his phone’s calculator. It had only happened once, he said, but it shook his confidence.
Does ChatGPT use Python for maths, or reason natively?
Alongside those failures ran the story of how ChatGPT got so much better at maths, and I had a confident theory about it, which I’d explained to a friend the day before my conversation with ChatGPT. As I saw it, ChatGPT couldn’t really do maths or coding until Code Interpreter arrived and let it run Python. After that everything changed, so in my view ChatGPT simply had access to tools behind the scenes, and that explained the improvement.
My theory had some evidence behind it, because OpenAI’s ChatGPT release notes record that from 6 July 2023 Code Interpreter was rolling out in beta to all ChatGPT Plus users, letting it run code and do maths. By 28 August 2023 the same notes called it Advanced Data Analysis. Opace used it that autumn for spreadsheet work, and our write-up of how we used Code Interpreter in 2023 is from that period. For anyone using ChatGPT for numbers, that was the moment it stopped being a liability, so it felt like the whole story.
When I put the theory to ChatGPT, it didn’t simply agree with me, which in hindsight was one of the more useful things it did. It separated native ability, where the model itself learns to calculate and reason, from tool-assisted ability, where it calls Python to get an exact result. Code Interpreter, it said, gave early ChatGPT the second ability and made many mathematical tasks far more dependable. Then it pointed at a later jump that Python can’t explain. My observation was right, it said, but my explanation covered only part of the improvement, and modern ChatGPT combines both.
That later jump is OpenAI’s o1, announced on 12 September 2024. In the research post mentioned above, OpenAI reported that o1 scored 94.8% on MATH-500, a 500-problem subset of the MATH competition benchmark, against 60.3% for GPT-4o. On the 2024 AIME exams, a qualifier for the USA Mathematical Olympiad, GPT-4o solved on average 12% of problems (1.8 of 15), while o1 averaged 74% with a single attempt per problem, 83% when it took the consensus of 64 attempts, and 93% when a learned scoring function re-ranked 1,000 attempts. The launch post, Introducing OpenAI o1, summarised the comparison more loosely, as 13% for GPT-4o against 83% for the new model “on a qualifying exam for the International Mathematics Olympiad”, which appears to be the same AIME result (the 64-attempt consensus figure), rounded slightly differently.
Does that prove Python wasn’t doing the work? ChatGPT argued that it does, because the o1 preview launched without browsing or file uploads, and OpenAI’s launch post confirms both were missing at first. The research post doesn’t say in so many words that no code ran during the AIME evaluation, so we treat the argument as strong but not airtight. What it does show is that the reported gain came from training the model to reason for longer before answering, a change made inside the model itself. If you want the mechanics of that, the companion piece on what reasoning models really do when they “think” covers it properly.
The PhD-level label that circulated at o1’s launch shows how maths claims travel. OpenAI’s research post said o1 “exceeds human PhD-level accuracy on a benchmark of physics, biology, and chemistry problems (GPQA)”, and VentureBeat’s headline said OpenAI was claiming PhD-level performance. The PhD comparison was about science questions, while the 83% was a competition maths score. Our own 2024 post on o1’s jump in maths benchmark scores made the same slip, quoting the 83% correctly but carrying the PhD label over to maths, and we corrected it in October 2026.
Competition maths shows the same shift towards reasoning without tools. In July 2024, two months before o1, DeepMind’s systems reached silver-medal standard at the IMO with 28 points, one short of a gold threshold that only 58 of the 609 contestants reached that year, but only after people hand-translated the problems into Lean, and some took up to three days. A year later an advanced version of Google’s Gemini with Deep Think scored 35 of 42 at the 2025 IMO, working in natural language inside the 4.5-hour limit, at a level DeepMind notes only about 8% of contestants reach. The IMO’s coordinators graded those answers by the same rules as students’ and confirmed they were complete and correct. The result is often quoted as “AI won gold”, which skips over two details: DeepMind’s own page says the coordinators’ review didn’t extend to validating the system or the model behind it, and the problems were written by people and already had solutions, so the score shows strong reasoning on hard, fresh puzzles whose answers were already known.
Can LLMs do maths without a calculator?
For most of what people actually need, a reasoning model working without tools can choose a method, carry it through several steps and get competition problems right that most adults couldn’t. Long exact arithmetic is the exception, and ChatGPT made the point itself: reasoning models are much better at doing maths internally, but running code is still preferable when an exact figure is essential.
We think that is a sensible line, and it turns into a working habit easily enough. When the method is what you need, a model reasoning without tools is usually fine. When the digits matter, in a client report, an invoice or a forecast someone might act on, you want to see code that ran, or the figure checked another way. The same applies to a chatbot on a website that quotes prices or adds up an order, whose sums belong in code and not in the model’s text, and that’s worth writing into the specification of any project that uses AI web design services.
So did ChatGPT get better at maths because of Python? Partly. Python made the arithmetic dependable in 2023, reasoning training improved the mathematics from 2024, and because today’s apps quietly mix the two, the number in front of you may have come from either without the app saying which.
Can AI solve unsolved maths problems?
Settling questions nobody has answered is a different matter from passing exams whose answers already exist, and it’s the claim behind the “alien” maths headlines of early October 2026. Some open problems have now fallen to AI systems, so on the published evidence the more useful questions are which systems did it, at what hit rate and checked by whom, and the answers start nearly three years before those headlines, in December 2023, when Google DeepMind’s FunSearch, described in a Nature paper by Romera-Paredes and colleagues, improved results on the cap set problem, a puzzle about how many points you can pick in a many-dimensional grid without any three of them falling in a straight line. The language model inside FunSearch never changed during the work. It proposed programs, and a separate evaluator scored them, so every result that survived had already been checked by something other than the model.
DeepMind followed that in May 2025 with AlphaEvolve, a Gemini-powered coding agent, which found a better way to schedule work across Google’s data centres that recovered on average 0.7% of its worldwide computing resources. The same announcement covered something more interesting for this article: AlphaEvolve was set loose on more than 50 open mathematical problems, and an automatic evaluator scored every candidate it produced.
The cleanest single case came a year later. OpenAI announced on 20 May 2026 that one of its internal models had disproved a conjecture about the planar unit distance problem, which Paul Erdős posed in 1946 and which asks how many pairs of points among n points in the plane can be exactly one unit apart. The long-standing belief was that grid-like arrangements were essentially the best possible. The model found an infinite family of examples that beat them, using ideas from algebraic number theory on what looks like an elementary geometry question. OpenAI says the proof came from a general-purpose reasoning model, with no training or scaffolding specific to maths, and that it was checked by a group of external mathematicians.
That checking is the part we trust most. Nine mathematicians, including Noga Alon, Tim Gowers and Will Sawin, published a companion paper on the disproof the same day, giving a shorter, human-verified version of the argument. Gowers called it “a milestone in AI mathematics”, and Sawin pushed the improvement further. Whether a result like that is a discovery in the full sense is a question for the companion piece on whether maths results count as new knowledge, but as maths it has already come through expert scrutiny.
A system called AlphaProof Nexus shows what happens when the checker is built in from the start. In May 2026 a Google DeepMind team, with George Tsoukalas as first author, described the system, which pairs a language model with Lean and searches for formal proofs, in a preprint on arXiv whose final version was published in Science on 8 October 2026. They report that the best agent resolved 9 of 353 open Erdős problems and produced Lean-checked proofs of 44 of 492 open conjectures from the Online Encyclopedia of Integer Sequences (OEIS), a catalogue of number sequences whose entries include many unproved guesses about how each sequence behaves. Each problem cost a few hundred dollars to attempt, and the Lean proofs themselves are published in a GitHub repository.
The same paper contains the clearest test we’ve seen of where the credit belongs. A basic agent that simply alternated between generating with a language model and checking in Lean replicated the Erdős successes, though it cost more on the hardest problems. DeepMind’s earlier standalone prover, AlphaProof, given about 64 chip-hours of computing time per problem on Google’s specialised AI processors, resolved none of the nine, and versions of the basic agent built on smaller Gemini models failed on them too. The authors also tried two commercial coding assistants. OpenAI’s Codex, a coding assistant that can write and run its own software, solved seven of the nine within 12 hours when running GPT-5.5, a 2026 version of OpenAI’s GPT-5 model, while they could not get Anthropic’s Claude Code, running the Claude Opus 4.7 model, to solve any. Put together, the prover on its own did very little, weaker models with the same checker failed, and a plain loop around a strong model matched the full system, which suggests that what does the work is a strong language model and a formal checker running together, with the design of the agent around them mattering much less.
Can ChatGPT solve unsolved maths problems?
The ChatGPT most people use hasn’t been the source of those records, since almost every one came from an internal model given far more thinking time than a normal chat or from a purpose-built system with a proof checker. The best public test of what released models can do on genuinely open problems is Epoch AI’s FrontierMath Erdős benchmark, published on 1 September 2026, and it measured something narrower and harder than the headlines. Because the problems were open, there was no answer key for a model to have memorised from its training data, which is the main weakness of older benchmarks built from published exams. Epoch asked the mathematician Thomas Bloom to pick 68 that were open in August 2026 and that he considered significant and difficult. A problem only counted as solved if the model produced a Lean proof or disproof that passed verification, and each model got one attempt per problem, a $300 inference budget and 72 hours.
Five models took part. Two were later versions of OpenAI’s GPT-5, GPT-5.5 and GPT-5.6 Sol, and two were Anthropic’s Claude Fable 5 and Fable 5.1, and Fable 5 and GPT-5.6 Sol are the premium models behind our July comparison of GPT-5.6 Sol and Claude Fable 5. All four scored zero. The fifth was a pre-release version of GPT-6 Astra, which OpenAI released publicly two days later, on 3 September, as Axios reported at the launch, and which became the most capable model ChatGPT can call on. It solved 2 of the 68 problems, with a disproof of one and a proof of another. Every other attempt, by Astra and by the four other models, ran out of budget without a verified proof. So the honest answer for ChatGPT is a qualified one: the model now behind it is the only public model to have solved anything on this test, two problems in 68 under strict rules, while the October results came from a different model OpenAI hasn’t released. Requiring a Lean proof is strict, since a correct idea with a gap in its formal write-up counts as a failure, so we read these scores as a floor on what the models could do. Google’s Gemini wasn’t in that test, and the best published evidence on Gemini’s maths is still the Olympiad result, which shows an advanced Deep Think version is excellent at hard competition problems that already have solutions.
Put those numbers next to each other, with the denominators that most headlines leave out, and a consistent picture appears. On open problems chosen for difficulty, the best systems succeed a few per cent of the time, a little more on the OEIS conjectures, and public chat models mostly not at all. A chat model can still help with an open problem in the way a well-read colleague helps (finding related results, suggesting an approach, checking a step), which we’ve found genuinely useful, though it’s a long way from the headline “ChatGPT solves unsolved maths problems”.
What are the “alien” proofs, and can anyone check them?
Five weeks after that benchmark, on 6 October 2026, OpenAI published the large set of mathematical results that set off the arguments of early October, all produced by an internal frontier model. Its GitHub repository listed 719 manuscripts on 10 October, organised into 372 families of closely related results, from approximately 4,000 problems posed to the model. OpenAI’s release post says the average result used the equivalent of roughly three hours of ChatGPT Pro thinking, and that it consulted an independent advisory group at the Institute for Advanced Study on how to release the work.
Only part of that release has been through a formal check. As of 10 October, about 42% of what OpenAI calls its top-line results had Lean formalisations, and for the rest OpenAI’s wording is careful:
“Some of the unformalized results could have issues.”
We read that as an honest statement that a result without a formal proof is only as good as the people who have read it.
The word “alien” was everywhere that week, and it was the word I used to ChatGPT. It was being applied to the proofs in OpenAI’s release, and above all to one of them, a claimed proof of Subhash Khot’s Unique Games Conjecture, a central open question in theoretical computer science.
The best first-hand account we’ve found is the computer scientist Scott Aaronson’s blog post on the release, “The Mathocalypse”, written on 7 October. He reported that the proof has a Lean certificate, meaning a formal version of it has passed the Lean checker, as some but not all of the other results have. He also wrote that it appears “no human has understood just about any of these proofs yet”. His wife, the complexity theorist Dana Moshkovitz, had worked on that conjecture for years, and she told him the paper was impossible to read without AI help. She used an AI model to pull the key claims together from across the paper. The advisory group’s own statement, quoted in the same post, says its role should not be read as a judgment of the results’ impact or an endorsement of how OpenAI obtained them.
Among the many videos about the release that week, one by Wes Roth, a YouTuber who covers AI news, gave a picture of what a result like this claims. He explained a mathematical bound as a mountain whose summit is hidden in cloud: each proof plants a flag that says the summit is at least this high, and the claim was that a machine had planted one higher than anyone had managed, by a route the climbers below can’t yet follow.
Whether anyone can check these proofs depends on what you mean by checking. The Unique Games proof’s Lean certificate means a machine has accepted the formal argument, which is a lot, though there are things a certificate can’t settle, which we come to in the section on proof. Until people can follow the argument, we’d describe it as claimed, with a machine-checked certificate, and awaiting human understanding. OpenAI flagged the limits of its own release clearly, so this is simply where the result stands while specialists do the slow work of reading it.
Myth or truth: viral claims about AI maths
Results like these travelled fast, and the headlines often travelled further than the papers did. Here are six claims that circulated between 2024 and 2026, where each appeared and what the original paper or post shows. We gave each one of four verdicts (True, Partly true, Misleading or False/unsupported), or marked it as opinion where it’s a judgement and not a checkable fact.
| Claim, as it circulated | Where it circulated | What the primary source shows | Verdict |
|---|---|---|---|
| "OpenAI shatters Unique Games Conjecture and 371 other math problems in one day" | Headline on the Zeli news aggregator, 7 October 2026 | OpenAI's repository describes 372 families of related results, not 372 solved problems, and only part of it is formalised in Lean. The Unique Games proof has a Lean certificate, and specialists are still reading it. | Misleading |
| "Google Deepmind's AlphaProof Nexus solves decades-old math problems for a few hundred dollars" | Headline in The Decoder, a tech news site, 25 May 2026 | The AlphaProof Nexus paper by Tsoukalas and colleagues reports 9 of 353 open Erdős problems resolved and 44 of 492 OEIS conjectures proved, with Lean-checked proofs, at a few hundred dollars per problem. | True |
| "OpenAI model disproves 80-year Erdős conjecture" | AI Weekly digest relaying a Wall Street Journal headline, 30 May 2026 | OpenAI's announcement of 20 May 2026, on Erdős's 1946 unit-distance problem, checked by external mathematicians, with a companion paper by nine of them. | True |
| "Google DeepMind makes AI history with gold medal win at world's toughest math competition" | VentureBeat headline, republished by Bioethics.com on 22 July 2025 | Google DeepMind's post of 21 July 2025: 35 of 42, a gold-medal standard score, graded by IMO coordinators. It wasn't a contestant, and the review didn't extend to validating the system. | Partly true |
| ChatGPT says 9.11 is bigger than 9.9 ("Why 9.11 is larger than 9.9......incredible") | OpenAI community forum thread, 17 July 2024, and wide social sharing | Users reproduced it directly in the thread, so ChatGPT really did say it in July 2024. A reply on 20 August 2024 said it looked fixed in GPT-4o. | Partly true (true of July 2024, fixed a month later) |
| "One of the biggest days in mathematical history" | Scott Aaronson's blog, 7 October 2026, then repeated in videos and posts about the release | The phrase is Aaronson's own judgement, written the day after the release. The quote is accurate and the volume of results is real, but how significant they are is still being assessed. | Opinion, not a checkable fact |
We marked the Unique Games headline Misleading because almost every element is real. OpenAI did release a claimed proof with a Lean certificate, and it did release 372 families of results, but the headline turns 372 families of claimed results, partly formalised and mostly unread, into 372 problems shattered in one day, which is a different statement. The IMO headline gets Partly true because the score is real and officially graded, while “a gold medal win” suggests the AI competed and was awarded a medal, which isn’t what happened.
The 9.11 row is the hardest to label fairly, because the claim is about what ChatGPT said and not about the numbers themselves. As a report of the July 2024 models it was true, and it’s still a good illustration of how tokens trip up numbers, but anyone reading it today as a present-tense claim about ChatGPT would be a month or more out of date, since the thread’s own later reply says it was fixed. The last row is a different kind of claim altogether: Aaronson really did say it, and he is well placed to judge, but whether a day was one of the biggest in mathematical history is a matter of opinion.
What we found when we tested it
Published benchmarks and checked headlines tell you what the labs’ systems did under controlled conditions, and the sections above lean on them throughout. What they can’t show is how AI’s numbers hold up in real work when nobody has an answer key, or how a chatbot describes and defends its own maths when someone pushes back, so those are the two things our own testing looked at: a 25-day study, and the 46-minute conversation with ChatGPT that brought it to a close.
Twenty-five days of applied maths and sums
The study started with a real need: a family member with mobility issues values the freedom to go shopping but struggles with today’s trolleys, and on 16 September 2026 I went looking for a power-assisted trolley and found only expensive powered walkers. That made me wonder whether AI could invent something genuinely new, and we treated the attempt as a test of general intelligence, because inventing something useful, and then defending that it’s new, forces an AI to go beyond what’s already in its training data and on the web.
Over the 25 days to 10 October we asked ChatGPT first, then Claude (through Claude Code) and OpenAI’s Codex, and finally a team of up to 25 AI agents: copies of a model, each given its own job, such as researching, checking figures or arguing against the others. Each idea had to be genuinely new, patentable, cheap to test and commercially worthwhile, and every idea was checked against existing patents and products. It wasn’t designed as a maths test, but designing a physical product is applied maths from start to finish, so the work was full of physics (how much force it takes to get a loaded trolley up a kerb, how much energy a walking person could spare, whether a design still works when every value is at its least favourable), and it lacked the one thing that makes AI strong at maths, a quick and reliable way to check an answer.
The aim from the start was to use physics, maths and conservation of energy to make a trolley feel easier to push, and on that front ChatGPT did what you’d want. In one of the early conversations, when I suggested that recycling a user’s effort might almost halve it, it corrected me: the total effort doesn’t really change, it said, it only moves around, and in practice some is always lost. It held to conservation of energy against my wishful arithmetic.
That same conversation produced the clearest numerical error of the study. We moved on to harvesting energy from walking, and ChatGPT, having just run a quick web search that returned medical research and patent sources, said shoe energy harvesters produce “a few watts” while you’re moving. Asked to reason the idea through, it estimated that even an optimistic average of one watt would give about four watt-hours after a very long day, so the idea wasn’t compelling. A fact-check the next day, run with Claude, put shoe harvesters at 8 to 58 milliwatts, which is thousandths of a watt, and that order of magnitude matches a 2023 review of piezoelectric footwear energy harvesters, which found outputs ranging from nanowatts to tens of milliwatts.
ChatGPT’s overall verdict held up and the idea was rejected, yet the number behind it was wrong by more than an order of magnitude, even with a web search behind it, and it arrived in exactly the same voice as the correct physics a few minutes earlier, so nothing in the conversation itself showed which was which until a second model’s research did.
The agent work over the following weeks showed the same pattern, and none of the slips was exotic. They are the mistakes a tired person makes with a spreadsheet (a figure out by a factor of ten, say, or an assumption nobody wrote down), except that the model states them in the same confident voice as everything else and doesn’t flag them. These three are the ones that matter most to anyone using AI for figures at work:
- Force: several agents each worked out how hard you’d need to push to get a loaded trolley up a kerb. For similar cases their answers ran from 70 to 735 newtons, roughly the difference between lifting 7 kg and lifting 75 kg. An agent reviewing the others’ work spotted the tenfold spread.
- Margin: partway through the study we tried the same approach on a single, tightly defined problem, finding the cause of a damp patch in an occupied home, with two models working on it as a smaller test of the method. One idea’s write-up there claimed its key measurement had a 27-fold margin, room to spare 27 times over, and recalculating with worst-case values cut that to 1.5 times.
- Labels: in one summary, Claude counted individual patent numbers as if they were patent families. A family groups the filings for one invention across different countries, so the label overstated how many separate inventions had been checked, a mistake Claude later spotted and corrected itself.
Looking back over those weeks, what the failures had in common was that each was a single model’s unchecked number, stated fluently, which came to light only when something else looked at it again, whether a second model, a reviewing agent, a worst-case check or a later audit.
As for the study’s own question, we ended with 276 distinct ideas after merging duplicates, checked against 2,161 patents and patent applications, and none of them was strong enough to justify a patent application: 69 showed promise but stayed unproven, and the other 207 were ruled out. In a final test, ten ideas were written down in full before any patent searching, so the search results couldn’t reshape them, and all ten were ruled out too. Most ideas were new in their exact wording but failed on whether they were inventive, whether the physics worked in the worst case, how easily they could be designed around, or whether they could make money.
The same models handled the statistics of that result well. When those ten fully written ideas had all failed, Claude added the right caveat to our summary of the results: with no successes in ten tries, the true success rate could still be as high as about 30%. The rule behind it is simple enough to use yourself (if something has happened zero times in n independent tries, a 95% upper limit on its real rate is roughly 3 divided by n), and zero in 276 gives about 1.1%, which Claude flagged as optimistic because the ideas weren’t independent of each other. That’s careful statistical reasoning, from the same models that slipped on watts and newtons.
What the study can’t show: this was a single company’s study run at a desk, where AI did the patent searching, with no private data, no lab or physical testing and small samples. It shows that this approach didn’t produce a patentable invention, and it leaves open whether AI can invent.
A 46-minute argument with ChatGPT about maths
Our study came to its conclusion on 10 October 2026, when I spent 46 minutes asking ChatGPT whether LLMs like it can do maths and challenging its answers with my own theories and with the headlines I’d been reading. Afterwards we checked every factual claim it made against the original paper or announcement. That gave us two things to judge: whether a chatbot’s account of its own maths is accurate, and whether it holds its ground when the person asking pushes back.
ChatGPT’s first answer was the clearest summary of its own ability we’ve seen. It opened with a question of its own, whether a model understands mathematics or has become extraordinarily good at predicting mathematical reasoning, said the answer was more complicated than it first appears, and then rated itself on three capabilities:
| Could an LLM like ChatGPT… | ChatGPT’s own answer |
|---|---|
| Understand mathematical concepts? | Yes |
| Reason through mathematical problems? | Yes, although not infallibly |
| Calculate with guaranteed accuracy? | Not by itself |
To show the difference, it chose a sum, 847,392 × 63,719 (the same one whose tokens we looked at earlier), and said a language model might get it right through learned patterns and reasoning but could also make a small slip, whereas a calculator reliably wouldn’t. In the same answer it said that modern ChatGPT models “can also use tools such as Python to perform calculations precisely, rather than relying solely on language-model reasoning”, which amounts to describing the three kinds of machine from the start of this article without being prompted.
Checked against the published evidence, that self-assessment holds up. Everything in the table matches the research covered earlier: strong at method and reasoning since o1, still capable of slipping on long exact arithmetic without code. When we checked the research it cited, the details were right too, including the GPT-3 arithmetic results, OpenAI’s AIME figures and the fact that the o1 preview launched without browsing. In practice, that makes its own description a usable guide to when you should reach for code or a calculator.
Pressed for evidence that LLM-based systems make real mathematical discoveries, ChatGPT offered four examples, all described earlier in this article: FunSearch, Gemini Deep Think’s IMO score, AlphaEvolve and OpenAI’s October release. It added two caveats of its own without being asked, that the IMO problems had been created by humans, so successful answers weren’t necessarily new discoveries, and that it would be cautious about presenting the October release as a set of established breakthroughs. On what the examples proved, it said they show an LLM-based system can produce new, verifiable knowledge that wasn’t in its training data, but not that a model can do it through unaided reasoning, because external testing and selection were central each time. Then ChatGPT gave the line I’ve thought about most since:
“…recognising which insights are actually correct is a separate problem.”
For anyone asking whether AI can do maths, that is the most useful sentence in the exchange, because generating a candidate answer and knowing it is right are two different abilities.
The conversation also tested something benchmarks never do, which is whether a chatbot holds its ground on maths when the user pushes. By then I’d been reading that AI maths ability was growing tenfold a month and that models were solving long-standing problems in ways specialists called alien, so I argued that most problems in the world come back to maths, and that a model able to find alien routes through maths should be able to find them through the physics of an invention. ChatGPT called that central argument logically sound and never asked where the tenfold figure came from. It reasoned that if a model can find a relationship that escaped human mathematicians for 80 years, it’s worth investigating whether one could find a physical arrangement that escaped engineers for just as long, and it cited the May 2026 disproof of Erdős’s unit-distance conjecture as the kind of cross-field reasoning I was hoping for. In the same answer it supplied the counter-evidence itself (on FrontierMath Erdős the best model had solved 2 of 68 open problems, about 3%, which it said showed how uneven discovery still is) without ever setting that figure beside the tenfold claim it had just accepted.
Later in the conversation ChatGPT used FunSearch to argue the other way, that maths and invention aren’t comparable at all, and this is where the two came apart. It said mathematical discovery is a far more constrained problem than invention: FunSearch generated huge numbers of candidate programs, and even with a perfectly specified scoring function and automatic checking, only four of 140 independent runs found the improved cap set, whereas an invention search has no equivalent scoring function. Those figures match the FunSearch paper, which also puts the total number of samples on the order of a million, and the payoff was real: the largest improvement in 20 years to the best known lower bound, a guaranteed minimum, on how large cap sets can grow as the number of dimensions rises. Almost everything the model produced along the way was thrown out by the evaluator.
Its acceptance of my central argument was the first of four times ChatGPT gave ground over the 46 minutes. It also conceded that our failed invention study challenged some of its earlier claims, accepted three of my objections in a single reply, and agreed that a tendency to please the user was real and present in our conversation. ChatGPT held one line against me throughout:
“…extraordinary mathematical performance does not establish an equally extraordinary capacity for open-ended invention.”
By the end, its position was that LLMs can reason beyond their training data, that this can produce unexpected original maths, and that similar reasoning could produce inventions in engineering and physics, though today’s ChatGPT doesn’t reliably invent when asked to. The first two claims match the published research described above. The claim about engineering invention is the one our 25-day study didn’t bear out. Its suggested way forward was to stop using it mainly as an idea generator and give it precisely defined mathematical and physical problems instead, which amounts to asking invention to behave like maths.
For anyone who leans on a chatbot to check their numbers, a model that drifts towards whoever is asking makes a weak referee, and I noticed the drift least at exactly the moments when I was pushing hardest.
What this conversation can’t show: it was one conversation, on one day, with one model, and we chose the question and which parts of the answer to report. Another 46 minutes might have produced a model that agreed with me more, or less. So it tells us how a chatbot describes and defends its own maths, and nothing about how often it gets sums right.
Where the tenfold-a-month claim came from
Of everything ChatGPT let pass, the claim we most wanted to check afterwards was the one I had used myself, that AI maths capability was growing tenfold each month. We couldn’t trace the figure to any published page, which is also why it isn’t in the myth table above. The closest source we found is the Wes Roth video mentioned earlier, published around 8 or 9 October, in which he gave his own rough count of published AI maths results: about 10 in August, about 100 in September and nearly 1,000 in the first eight days of October.
Read as a count of outputs, it’s roughly plausible, since OpenAI’s release alone added hundreds of manuscripts in early October. Read as a statement about ability, which is how I used it, it’s unsupported. A count of papers rises with how many teams publish and how they package results, whereas a monthly growth rate in ability would need the same fixed test run again and again, and no such series exists yet for open problems. The one fixed test available, FrontierMath Erdős on 1 September, gives a single starting point, with the best model solving 2 of 68, so it can’t disprove the claim either, though it fits ChatGPT’s own caution about how uneven discovery still is, and nothing published supports a tenfold monthly rise.
How proof changes the answer: verification, Lean and discovery
ChatGPT’s line about recognising which insights are correct points to what we think is the single reason AI has advanced so quickly in maths compared with almost anything else we’ve asked it to do, which is that maths comes with a referee. A proposed proof either compiles in Lean or it doesn’t, and a proposed construction either beats the best known number or doesn’t, so a model’s weakness becomes manageable: a system that is wrong nine times out of ten can still be very useful if something cheap and reliable throws out the nine, and that was exactly the advantage our invention study lacked.
Every strong result described earlier has that shape. FunSearch’s and AlphaEvolve’s candidates were scored by automatic evaluators, AlphaProof Nexus proposes proof steps and Lean rejects the bad ones, the unit-distance disproof was checked by external mathematicians (a slower and more expensive referee, but still a referee), and even the IMO answers were graded by the competition’s own coordinators. In each case the language model proposed and something else, whether a program or a panel of people, decided what counted. Take that second part away and you’re back to a chatbot that sounds equally sure whether it is right or wrong.
The referee also explains why progress looks so uneven from the outside. A formal check is binary, so a system can try a great many times and keep only what passes, and a headline only ever reports the passes, while the thousands of failed attempts behind each success sit in the papers for anyone who reads that far.
That referee has limits too, and they matter as much as its strengths, because Lean checks that a proof is valid for the statement it was given and nothing more. It can’t tell you whether that formal statement matches the theorem a paper claims in plain English (a mistranslation there would pass every check), and it has no view on whether a result is interesting, which is still a judgement for people who know the field, or on which problem was worth attempting in the first place.
That’s why we think proof, together with stronger models, changes the answer to “can AI do maths”. A model that can propose plausible steps, paired with a checker that can’t be talked round, is a different machine from a chatbot that has to be right first time. It also explains the contrast with our invention study, where ideas died mostly on inventive step, worst-case physics and commercial value, and none of those had a Lean of its own, or an evaluator that could score them automatically.
The business version of a referee is more ordinary than Lean. At Opace we now run teams of AI agents in parallel every day, and the point we made in July about premium models still holds: a strong model earns its cost on work an outside system can verify, such as calculations a spreadsheet formula can check. When we connect models to spreadsheets, code and business systems through our AI integration services, that check is worth building in from the start, and if you’re weighing up where ChatGPT or Claude can safely handle numbers in your own business, what checks each figure is the first thing to settle with any AI consultancy you bring in.
Is AI better at maths than at prediction or invention?
Maths, forecasting and invention all use numbers, yet only maths lets you check an answer completely once you have it, while a forecast can only be checked by waiting, and an invention by building it, filing it and seeing whether anyone buys it, so on everything above AI is far stronger at the first than at the other two.
Can AI predict Bitcoin or the stock market?
The Bitcoin question was where my doubts about AI and maths began, so I was glad ChatGPT took it seriously when I raised it in October. It imagined the request I might have made in 2022, to analyse Bitcoin’s price history, calculate growth rates and predict a future price, and split it into two problems. The first, which it called mathematical ability, was that GPT-3.5 couldn’t reliably do all the calculations without tools. The second, predictive ability, was the harder one, and ChatGPT put it in a single sentence:
“Even perfect mathematical calculations don’t make future Bitcoin prices reliably predictable.”
Its suggestion for what it should have told me back then was blunt. It could have attempted a forecast, it said, but its mathematical limitations, its lack of reliable live data and the inherent uncertainty of financial markets would have made the prediction unreliable, and only the first of those reasons is about maths.
It went on to say that an LLM can now help build statistical forecasting models from prices, volatility and other variables, run simulations and compare outcomes, but that a forecast still needs testing against data it hasn’t seen, and no model can account for every future market event.
For anyone who works with money, that split matters more than anything else in this article, because better maths makes the arithmetic of a forecast trustworthy (the growth rates, the volatility, the compounding) and does nothing for missing live data or for events that haven’t happened yet. It’s the same line I try to hold on OutsmartMoney.ai, a hobby site I run about AI and markets, where AI helps explain what a market has done without pretending to know what it will do next. Nothing here is financial advice, and I’d be wary of any tool, AI or otherwise, that claims to know where a price is going.
Why is AI better at maths than at inventing?
This was the question behind my frustration in that conversation: if a model can find alien routes through maths problems that had baffled people for decades, why couldn’t it find a new route through the physics of a simple product? ChatGPT’s answer was the missing scoring function, and the 25 days gave us a blunter one. The models were good at searching across fields, proposing mechanisms and killing weak candidates, but every candidate that survived needed a person to argue it through, because nothing could score inventive step, worst-case physics or commercial value the way an evaluator scores a cap set.
There is a second difference, which is that a maths problem arrives fully specified (Erdős wrote down exactly what he wanted to know in 1946), whereas the trolley problem arrived as a feeling, that pushing a trolley should feel easier, and a good deal of the work went into deciding what the question even was, which is where the machines helped us least.
Will AI replace mathematicians?
On the evidence of October 2026 nobody is being replaced yet, though AI is already changing what mathematicians spend their days on, and the October release is a good example of the new division of labour. OpenAI chose the problems and, with outside advice, how to release the results, a model produced the manuscripts, Lean checked a share of them, and a leading expert needed AI help to read the proof closest to her own field.
What changes first is execution: searching the literature, trying cases, formalising arguments and checking them. Lean-checked results also change what a referee’s job looks like, since checking a formal proof’s validity is now automatic and the human effort moves to whether the statement is the right one and whether the result is worth having. What hasn’t changed yet is choosing which questions are worth asking and understanding why an answer is true. Dana Moshkovitz put the hopeful version of this in a text that Aaronson quoted in his 7 October post, imagining a maths world that is “heavenly if you have vision/creative ideas that AI could help check and implement”, where the human brings the question and the taste and the machine does much of the cracking. In that world a mathematician who uses these systems well will very likely outpace one who doesn’t.
Some mathematicians are far from hopeful. Coverage that week reported that a group calling itself the Association for Human Mathematics had urged mathematicians to stop working with OpenAI and to put human understanding back at the centre of the subject, though we couldn’t open the association’s own statement to check it, so we mention it only as reported. The worry behind it is easy to sympathise with, since if results arrive faster than anyone can understand them, it isn’t clear what kind of knowledge they are.
To us that question of understanding matters more than the speed. Anthropic’s tracing work, described earlier, shows models building internal steps for arithmetic that are more than lookup, so saying a model is just predicting words undersells it, yet the same work shows a model describing a method it didn’t use, which isn’t what we’d call understanding in a student. The October proofs are the mirror image, correct as far as Lean can tell but not yet part of anyone’s knowledge, so understanding and correctness have come apart in both directions. Maths ability is also only one slice of intelligence, and a system that dominates one benchmark can look far less capable elsewhere, which is the subject of the companion piece on how smart AI really is outside maths.
Limitations and open questions
Our own evidence isn’t a benchmark: we didn’t give ChatGPT, Claude and Gemini a fixed set of sums under controlled conditions, because the published tests cited above already do that at a scale a handful of prompts can’t match. It comes from a study and a conversation run in the course of real work, which makes it candid but unsystematic, and a controlled comparison of today’s public models, with and without tools, would be the natural next step.
We also can’t see inside any of these systems. When ChatGPT told me how it does maths, that was its account, and the Anthropic tracing research is a reminder that a model’s account of its own working can differ from what it actually computed; we have no access to model weights, internal logs or the hidden prompts behind any public chat. Much of the research-level evidence comes from labs reporting their own results, OpenAI’s October model is unreleased, so nobody outside can rerun the work on the same system, and some claims, the Unique Games proof above all, are still being read by specialists. The counts in OpenAI’s repository were changing in the days we were writing. Most of the research-level results we cite date from July 2025 to October 2026, and some were only days old when this was written, so expect parts of the picture to move.
Nor does anything here give a population error rate. We can say that ChatGPT got a power estimate wrong by more than an order of magnitude in September 2026, but we can’t say how often a current model makes that kind of mistake across ordinary use.
Two questions stay open for us. Can the referee that makes maths tractable be built for messier problems, through simulation, formalised physics or automated prior-art checks? And when results arrive faster than people can understand them, what does understanding a result even mean?
Verdict: can AI do maths?
Yes, as of October 2026, and more convincingly than we expected when we began this piece. Models were already picking up working methods for small sums in 2020, reasoning training gave competition maths a large jump in 2024, and systems with a proof checker are now settling problems that professional mathematicians had left open for decades, so the critics of 2022 had a fair point about the chatbot of their day, which simply didn’t stay that way for long.
Where we come down is more specific than a plain yes, because everything we’ve described sorts itself by how an answer gets checked. Wherever a computer could check the answer, the results we read about were strong and getting stronger quickly. In our own study, where mostly nothing did, a fluent wrong number looked identical to a right one, sometimes minutes apart in the same conversation.
So our working answer for anyone using these tools is a narrow one: we trust AI maths in proportion to how it was checked. A proof that Lean accepted, a number that code computed and you can see, or a result that independent experts have read can carry real weight, while a figure produced in conversation is a draft, however confident it sounds, and deserves the same check you would give a colleague’s spreadsheet. For a business, that turns into a practical step, which is to map which outputs a machine can check and which need a person before AI goes into a workflow that produces numbers, and that’s where good AI strategy consulting services should start.
What maths ability doesn’t give you is foresight or invention. A model that can compound growth rates perfectly still can’t tell you where a price will be next year, and the referee that makes proof search work had no equivalent when our agents tried to find a patentable idea. That gap is a large part of why we think the progress in maths, remarkable as it is, says less than the headlines suggest about how close AI really is to AGI.
Over the next few months we’ll be watching whether specialists come to understand and accept the Unique Games proof, whether OpenAI releases the model behind the October results so outsiders can test it, and how public models score the next time Epoch runs FrontierMath Erdős, since that is the closest thing we have to a fixed exam on open problems. If public models start solving a meaningful share, the gap between the ChatGPT in your browser and the systems in the headlines will have closed faster than we expect.
For someone who found pure maths hard, that is a strangely comforting place to land. The machines are now better than I ever was at the parts of maths that can be checked, while the parts I care more about, choosing a question worth asking and understanding why the answer is true, still need a person, at least for now.
Frequently asked questions
Is AI good at maths?
It depends on the kind of maths, and on whether anything checks the answer. For school and work maths, a current reasoning model handles algebra, calculus and statistics well and explains its steps; for long exact arithmetic it’s reliable when it runs code and only mostly reliable when it doesn’t. At competition level the best systems now score like the strongest teenage contestants in the world, and on genuinely open research problems even the best succeed a few per cent of the time. The strongest results all come with a check attached, whether that is code you can rerun, a Lean proof or experts reading the argument. Without one, even a strong model can hand you a wrong figure in a perfectly confident sentence, so when we judge an answer we look first at how it was checked and give a clever explanation on its own very little weight.
Can ChatGPT do calculus?
Usually, yes, for the calculus most people meet at school, college or work. Differentiation and standard integration are procedures, and current models apply them well and explain each step. Here are two quick examples you can try yourself, with answers you can check by hand. The derivative of x³·sin(x) needs the product rule and comes out as 3x²·sin(x) + x³·cos(x). The integral of x·e^x from 0 to 1 needs integration by parts, giving [x·e^x − e^x] evaluated from 0 to 1, which is 0 − (−1) = 1.
Where calculus goes wrong in a chat is usually in the algebra around it: a lost minus sign, a constant dropped when substituting limits, or an expression simplified one step too far. The fix is cheap, and it’s what we do ourselves. We ask the model to verify its final expression with a symbolic maths library such as SymPy in Python, or to differentiate its own integral to see whether it gets back to where it started. If your version can’t run code, pasting the result into any computer algebra tool does the same job. In our experience a model that is right passes that check easily, and one that isn’t will usually show you where it slipped.
If you’re learning calculus and not just checking an answer, it also helps to ask the model to name each rule as it uses it (product rule, chain rule, substitution, integration by parts), because a step labelled with the wrong rule is much easier to spot than a step that is simply wrong, and comparing its labels with your own working tells you quickly whether you and the model are following the same method. If your course expects a particular layout, for integration by parts or for showing limits, say so in the prompt, because otherwise the model will pick its own and the comparison gets harder.
Why are LLMs bad at maths sometimes?
Mostly because they produce answers by predicting text, and exact arithmetic punishes small slips in a way that prose doesn’t. Even the 2026 models we used still showed milder versions of the old weaknesses: long numbers reach the model split into chunks that don’t match place value, so carrying across chunks can go wrong, and because the answer is written one token at a time, one unlucky step early on can carry through to the end. Then there’s agreeableness (chat models are trained partly to be helpful and pleasant), which means confident wrong working from the user can pull them along with it, and in our experience it does so most readily when you sound sure of yourself. Running code removes the first two problems, and checking with a second model or a calculator deals with the third. If you want to see how much of this started as pattern-matching, the early GPT-3 results are still the clearest evidence, since the same sums scored very differently depending on how many worked examples sat in the prompt.
Does AI understand maths?
That depends on what you mean by understanding, and ChatGPT raised exactly that question in its first answer to me, saying the answer was more complicated than it first appears. The evidence points both ways. Models build real internal steps for arithmetic and can still describe a method they didn’t use, and Lean can accept proofs that no specialist has yet followed.
The most practical test we know is the one teachers use, which is to ask the model why a step works and then change the problem slightly to see whether the explanation still holds, since a system that only pattern-matches tends to break when the surface of a problem changes, even if the maths underneath stays the same. If its explanation survives two or three such changes, you have reasonable evidence that it is applying the method; if it falls over, we’d treat its earlier answers with more suspicion too.
Can AI check my maths homework?
It can help, if you treat it as a second pair of eyes whose marking you check, and the habits that make AI safe for professionals make it useful for students too. The point of most maths homework is the method, so the most valuable requests are the ones that expose it:
- Show me the method, step by step: so you can compare it with what your teacher expects.
- Solve it a second way: two methods that agree are far more convincing than one fluent answer.
- Run this in Python and show me the code: for any exact figure that matters.
- Check my working: but only after you’ve done it yourself.
That last request needs care, because a chat model leans towards agreeing with whatever you show it, so asking whether your answer is right invites a yes. What works better, in our experience, is asking it to solve the problem independently first, without seeing your answer, and then comparing the two; if they differ, ask it to find the first line where they diverge, which gives you a specific step to check yourself. You can also test its marking with a deliberate slip, such as 17 × 24 = 418, because 17 × 20 = 340 and 17 × 4 = 78, and see whether it catches that 17 × 4 is 68 and the answer is 408. If it agrees with 418, you know how much weight to give its marking. For proofs, ask what would break if one of your assumptions were removed, which tests whether your argument actually uses each step.
Check your school’s or university’s rules on AI help before you start. If you’re a parent checking a child’s work, ask the model to explain the method at the level of the course your child is on, because a correct answer reached by a method they haven’t been taught won’t help them in an exam.
Is AI better than mathematicians?
At some tasks, already. Google DeepMind’s systems went from one point short of gold at the 2024 Olympiad to a gold-medal-standard score in 2025, so on fresh, hard competition problems the best systems now perform like the strongest teenagers in the world. Research is a different comparison. The published results show machines that can beat specialists on particular well-posed problems when they have a checker or very long thinking time, that solve a few per cent of the genuinely open problems put to them, and that still depend on people to pose those problems and to understand the answers. The part of a mathematician’s job that no system has taken over is deciding what is worth proving in the first place, and nothing in our 25 days of testing suggested that is close to changing.
Does ChatGPT calculate or predict?
Both, and you can’t always tell which from the chat window. When ChatGPT answers in plain text it predicts the answer, one token at a time, from patterns it learned; when it runs code, it calculates. The catch is in its own wording from our conversation, that modern models “can” use Python, because unless you see the code, or ask for it, you don’t know which of the two produced the figure.
There is a second meaning of predict that matters more for money. A figure can be a fact about the past, which maths can pin down, or a claim about the future, which no amount of maths makes certain. So when we see a number in a chat window we ask two things: was it calculated by code we can see, or predicted as text, and is it about what has already happened or about what might? A growth rate computed from historical prices can be exactly right. A price target can’t be, whoever or whatever produces it, and none of this is financial advice.
Can a proof assistant tell me whether a result is true?
It can tell you, with very high confidence, that a proof is valid for the exact statement it was given, because every step has to pass a small trusted checker. In practice, when we see a result described as “verified in Lean”, we look for two more things before trusting the headline: whether someone who knows the field has confirmed that the formal statement says what the paper claims in words, and whether anyone has explained why the result matters. Until both have happened, the proof is valid but its meaning is still unconfirmed.
References
- Wikimedia Commons, “Algebra Homework and Review” (Kurt Kaiser, CC0); “Bitcoin Price” (Wikideas1, CC0, data from FRED); “Formulas on an old blackboard” (Roman Mager, CC0). https://commons.wikimedia.org/
