Can AI create new knowledge? What 25 days of testing taught me

Can AI create new knowledge, or is ChatGPT a very good search engine with excellent grammar? That is usually what people mean when they ask. If a language model can only rearrange what it read in training, it is a useful assistant and not much more. If it can work out things nobody knew, or find facts nobody has seen, then research and a good deal of business will change with it. The question has become pressing because the claims have grown so quickly: in May 2026 nine mathematicians verified a counterexample to an Erdős conjecture that an OpenAI model had produced, and in early October OpenAI released about 720 maths manuscripts in one go. It is also one of the tests underneath the wider argument about how far we are from AGI, because inventing something useful, and being able to show it is new, means going beyond what is already in a model’s training data and on the web.

We came to the question with evidence of our own, because a family member of mine has mobility issues and values the freedom to go shopping but struggles with today’s trolleys, and when I went looking for a power-assisted trolley in September 2026 I found only expensive powered walkers. That made me wonder whether AI could invent something genuinely new. So over 25 days, from 16 September to 10 October 2026, we at Opace put the challenge to ChatGPT, then to Claude (through Claude Code) and OpenAI’s Codex, and in the final stage designed a team of up to 25 AI agents for the same job. Every idea had to be new, patentable, cheap to test and commercially worthwhile, and the ideas that survived a first read were checked against existing patents and products. The models produced 276 distinct ideas, and none was judged strong enough to justify a patent application.

Our short answer is yes, in two of three senses: language models recombine ideas and reason their way to conclusions nobody wrote down, and discovery systems with automatic checkers have produced new, verified maths. What a chatbot cannot do alone is establish a new fact about the world, which still takes an observation, a measurement or an experiment.

Below we explain what new and knowledge mean here, look at the AI systems that genuinely have discovered things and what made them work, and then set out what our own test found, what ChatGPT made of the result when we put it to the model, and what the result can and can’t show.

Which AI, and what counts as new knowledge?

AI covers a lot more than a chat window. It includes recommendation engines, vision systems, robots, protein-structure models and purpose-built discovery systems that pair a model with an automatic judge. When most people ask whether AI can create new knowledge, though, they mean a large language model (LLM) such as ChatGPT, Claude or Gemini answering a prompt. That is the case our test covers, and we say where a complete system changes the answer, because on this topic it changes it a great deal.

Much of the framework in this article comes from the conversation that closed our study, a 46-minute debate with ChatGPT on 10 October in which we put the whole result to the model (we report its verdict on our failed invention search after the study itself). Early in that conversation it accepted that it had cited FunSearch and AlphaEvolve, two of Google DeepMind’s discovery systems, “without sufficiently distinguishing the abilities of the underlying LLM from those of the complete discovery system”. Those systems show that a model inside the right machinery can make original, verifiable discoveries; they don’t show that a general-purpose chatbot can invent a useful, patentable device on request, and that distinction runs through everything below. Because the concession came from the model being tested, we treat it as a claim to check against the published record, which is what the sections on discovery systems do.

Three kinds of new: recombination, reasoned discovery and empirical discovery

Early in the same conversation, ChatGPT split new into three kinds:

• Recombination: a new arrangement of existing ideas.
• Reasoned discovery: deriving a genuinely new conclusion or solution from what is already known.
• Empirical discovery: establishing something new about reality, which generally needs fresh evidence, an experiment or an observation.

Its view was that a language model can manage the first two without outside tools and that the third is where its limits start to matter, which is the claim most of this article sets out to test.

An everyday example makes each one concrete: a restaurant that puts a Korean sauce on a British pie has recombined. A student who works out that a chess position is a forced win has reasoned their way to something nobody wrote down, without learning a new rule. A biologist who finds that a gene controls a particular enzyme has discovered something about the world, and only an experiment could have told her. The AI parallels are close: a chatbot suggesting a product mash-up, a system such as FunSearch finding a new mathematical construction that an evaluator can score, and a robot laboratory confirming what a yeast gene does.

The three-way split also settles an argument that runs through much of what is written on this question. One camp says AI obviously creates knowledge, because it produces true, justified statements no one has made before. The other says AI is backward-looking, bound to the consensus of its training data, a case the strategy researchers Teppo Felin and Matthias Holweg made most sharply in Strategy Science. Their argument rests on what they call data–belief asymmetries: AI’s data-based prediction, they write, “is largely backward looking and imitative”, while human theory-based reasoning lets people intervene in the world and run directed experiments that generate new data. Their worked example is heavier-than-air flight, where the Wright brothers’ belief that powered flight was possible ran against much of the expert opinion of their day, so a model trained only on the writing of that period would most likely have sided with the consensus, and the brothers settled the matter by building and flying. The first camp is describing recombination and some reasoned discovery, and the second is describing empirical discovery, so each is right about a different kind of new and the two camps mostly talk past each other.

Recombination, deduction and observation are old categories in the philosophy of science, and ChatGPT did not invent them, but arranging them into a test for this question, at the moment we needed one, was new to us and genuinely useful, which makes the framework itself an example of recombination doing its job well. We use it because it holds up against the discovery systems and papers covered below (a chatbot proposing it would not have been enough on its own), and because it is a reminder that a new concept, a new way of sorting what we already know, is a different kind of novelty from a new fact.

What counts as knowledge?

The philosophers’ starting point is the old justified true belief analysis. The Stanford Encyclopedia of Philosophy’s entry on the analysis of knowledge, last substantively revised on 21 January 2026, sets it out as three conditions: the claim is true, the person believes it, and they are justified in believing it. Edmund Gettier showed in a 1963 paper that those three are not quite enough, but for our question the third condition matters most, because a sentence only becomes knowledge once something has checked it.

Language models get awkward at exactly this point, because a model trained for next-token prediction learns what tends to follow what. Frequency and truth usually travel together, which is why the answers are so often right, but a rare true answer can lose to a common false one, and a confident fabrication, the thing people call hallucination, looks exactly like a fact on the screen, so fluency on its own can’t count as justification. Whether the model believes anything is a question we leave to the philosophers. Noam Chomsky, Ian Roberts and Jeffrey Watumull made the sceptical version of this case in The False Promise of ChatGPT, a guest essay in The New York Times on 8 March 2023, arguing that such systems predict without explaining.

When we ask whether AI can create new knowledge, then, we mean something specific: a statement that is new, that is true, and that has passed a check a sceptical person would accept. A proof checker, an automated evaluator, a reproduced experiment and a patent examiner are all checks of that kind, while a model’s own sense that an idea is novel, as our study showed many times over, is not.

Running Opace since I founded it in 2008 has taught me a business version of the same distinction, in which new to me, new to the market and new to the world are three different bars. A chatbot clears the first one many times a day, the second needs some market research, and the third needs a search, a test and usually another person, which is the bar our study set.

Can AI create new knowledge, or only rearrange what it knows?

My own starting point was a premise I suspect many people share, and when the study was over I raised it with ChatGPT much as I’d always thought of it: “Reasoning is great, but you can only reason based on what you know.” If what you know grows over time, you can then reason from the new material, but on that view a model whose knowledge stopped at the end of its training can never truly go beyond it.

ChatGPT disagreed with half of it, arguing that a language model can produce genuinely new ideas and solutions that were not in its training data through reasoning alone, and it put the reason in one line:

“You don’t necessarily need to acquire new knowledge to create new knowledge.”

Its first example was a mathematician who knows ten rules: those rules might support thousands of conclusions, some so complicated that nobody has worked them out, and drawing one out adds no new external information, since it is an implication of what was already known. Its second was chess, where you can know every rule of the game without knowing every possible game, so reasoning about the rules can find a winning sequence nobody has ever played.

That challenge seems fair to us, because my premise treated what you know as a fixed pile of facts, when a small set of rules can contain far more than anyone has drawn out of it. It holds for anything defined by its rules, which includes mathematics, logic, games and a good deal of engineering calculation.

My instinct still held somewhere else, though, and ChatGPT drew that line too. Suppose you ask a model whether an undiscovered planet orbits a particular star. It can apply astrophysics and estimate the odds, but no amount of reasoning can settle a fact about that star that the evidence leaves open. The same applies to a future Bitcoin price, which depends on decisions nobody has made yet. Its summary of the boundary is worth quoting in full:

“Logical reasoning can reveal new implications of what is known. Empirical discovery generally requires evidence about the world. This is not a special limitation of LLMs. It’s a limitation of deduction itself.”

It would be easy to read the failure of our invention search as proof that machines are uniquely limited, but on this point they aren’t, since the boundary ChatGPT described would stop me, or a physicist, just the same if we had only the same evidence to work from.

There is a harder version of the question: could an AI keep generating hypotheses, testing them, storing the results and reasoning from those results, without people supplying each new idea? ChatGPT’s view was that today’s discovery systems already show parts of this, but rely on external evaluation and only work in particular domains. We think that qualification matters more than anything else in its answer, because the systems that have discovered things are the ones with something outside the model to say whether a candidate was right.

Can an LLM create new knowledge from what it already knows?

Yes, in the reasoned-discovery sense, and the next section shows it happening, since a system does not need to have met a solution before in order to find it. The harder part is proving that a given result really was new, because nobody can list everything a model saw in training. The major labs don’t publish their full training sets, and leakage from public tests is a real risk: when one research team wrote a fresh set of grade-school maths problems in the style of a popular benchmark, several model families scored up to 8% lower on the new problems, which the authors read as evidence of systematic overfitting to the old ones.

The reason it is so hard to tell lies in how these systems are built: a language model turns its training into an answer by predicting likely continuations from patterns learned across a vast amount of text, and we explain how a language model turns training into an answer in an older, simpler piece. When a model finds an idea unfamiliar, all that tells you is what its training happened to miss.

AI-generated work also inherits the shape of its sources. Opace has written about AI that reproduces the bias in its training data in image generation, and the same pull operates on ideas. In practice the most probable answer tends to be the most written-about one, which usually means somebody has had it already, so a business asking a chatbot for a product idea should expect the first suggestion to exist somewhere before anyone spends money on it.

Can AI go beyond its training data?

ChatGPT separated three places where a model can hold information, and the split is useful for anyone trying to understand what these tools can and can’t learn:

• The weights: what was learned in training, which does not change during an ordinary conversation.
• The context window: the conversation and anything pasted into it, held only for that conversation.
• An external store: a memory feature or a database that can keep findings for later without touching the weights. Before its launch we wrote about the fuller conversation memory we hoped GPT-5 would add, which is this third layer.

Whether AI can learn new things depends on which of those layers you mean. In the context window it obviously can, since a model can read a document you give it, derive something from it and use the result in the next step. With fixed weights and a well-checked store of findings, a model could grow far more knowledgeable over time, and ChatGPT compared that to a researcher who studies for 30 years and knows vastly more at the end, although their underlying ability to reason hasn’t necessarily risen in proportion. It also warned that unlimited memory is not unlimited intelligence, because a system can build ever more elaborate conclusions on one wrong assumption. Then it posed the scaling question we find hardest of all: take a model whose weights are fixed for good, give it unlimited memory and a reliable way to check its discoveries, and would it plateau or accelerate? Such a system might keep gaining knowledge while its ability to reason stayed where it was, which could still be very useful, but whether its rate and reliability of discovery would keep improving is an empirical question nobody has answered, and it sits close to the centre of the AGI debate.

ChatGPT also separated two claims that are easy to run together: Claim A, that a language model can discover something genuinely new that its makers never taught it, for which it called the evidence strong, especially in mathematics, and Claim B, that a model can become steadily more intelligent simply by thinking and remembering, which it called a much bigger claim. Our 25 days tested a narrow and demanding slice of Claim A, namely whether a model could discover something untaught that was also true of the physical world and useful enough to patent. The memory side of all this, and whether AI remembers what it discovers, has its own article in our series on AI and AGI.

Can AI discover new things? The systems that already have

The strongest evidence that AI can produce new knowledge comes from systems, and the pattern in every case is the same: a model proposes, something outside the model checks, and only what passes the check gets called a discovery. We grouped the main examples by what did the checking, whether a laboratory, a piece of software or a team of mathematicians, and took them oldest first within each group.

When the lab is part of the loop

The oldest example we found had a laboratory doing the checking: the Robot Scientist Adam, described by Ross King and colleagues in Science on 3 April 2009, “autonomously generated functional genomics hypotheses about the yeast Saccharomyces cerevisiae and experimentally tested these hypotheses by using laboratory automation”. The team then confirmed Adam’s conclusions through manual experiments and formalised the work so that it linked 6.6 million biomass measurements to their logical description, which gives a sense of how much of the discovery was measurement. That is empirical discovery by a machine, and it worked because the machine could run its own experiments.

The most celebrated case is AlphaFold2, and the Nobel committee chose its words with care. The 2024 Nobel Prize in Chemistry press release, dated 9 October 2024, gave half the prize to Demis Hassabis and John Jumper “for protein structure prediction”, a word that leaves the measuring to laboratories, for an AI model that solved a 50-year-old problem. Since presenting AlphaFold2 in 2020, it says, they had been able to predict the structure of virtually all the 200 million proteins researchers have identified, and the tool had been used by more than two million people in 190 countries. AlphaFold learned from decades of laboratory-measured protein structures, and its predictions are still judged against new measurements.

Google DeepMind’s materials model, GNoME, shows why the counting unit matters, because the Nature paper of 29 November 2023 reported 2.2 million predicted crystal structures that would be more stable than previously known compounds, 381,000 of them newly discovered stable materials, and said that 736 had already been independently made in laboratories. Those are three different claims: predicted, predicted stable and made. Anthony Cheetham and Ram Seshadri reviewed the work in Chemistry of Materials in April 2024 and found “scant evidence for compounds that fulfill the trifecta of novelty, credibility, and utility”, and both papers can be right, since the model produced a vast map of candidates and whether many of them deserve the word material is a question for synthesis and use.

The same split between a system’s proposal and the evidence that confirms it runs through drug discovery and medicine, where the stakes are higher and the timelines longer. We look at whether AI can make scientific discoveries in medicine in a separate article.

FunSearch and AlphaEvolve: models inside discovery systems

A fortnight after the GNoME paper, Google DeepMind published FunSearch in Nature on 14 December 2023, and here the checker was software. FunSearch paired a pretrained language model with an automated evaluator. The model wrote short computer programs, the evaluator ran and scored them, and weak programs were discarded while strong ones went back into a pool that shaped the next round. DeepMind’s announcement called it the first time a new discovery had been made for challenging open problems in science or mathematics using language models, and described the cap set result as “the largest increase in the size of cap sets in the past 20 years”. A cap set is a collection of points with no three in a line, a problem that has kept mathematicians busy for decades. The author list included the mathematician Jordan Ellenberg, a reminder that people chose the problems and judged what the results meant. The same post describes the evaluator as the part that protects against hallucinations and incorrect ideas, and that is where the credibility came from.

The paper is candid about how hard the search was: its results came from something like a million samples, and for one of the cap set constructions “only four out of 140 experiments” found a cap set of size 512 in eight dimensions. Even with a perfect scorer a new result was rare, and we find that reassuring, because the system’s credibility rests on the evaluator rejecting almost everything the model wrote, a job an ordinary chat window leaves to the user.

Google DeepMind’s AlphaEvolve, announced on 14 May 2025, scaled the same idea with Gemini models and automated evaluators that verify each answer. DeepMind reported three kinds of result:

• a data-centre scheduling heuristic, in production for over a year, that “continuously recovers, on average, 0.7% of Google’s worldwide compute resources”;
• a way to multiply 4×4 complex-valued matrices with 48 scalar multiplications, improving in that setting on Strassen’s 1969 method, published in Numerische Mathematik as “Gaussian elimination is not optimal”;
• improvements on about a fifth of the more than 50 open mathematical problems it was tried on, and a match for the best known answer on about three quarters.

Every one of those results came with a scorer (the scheduler’s recovered compute, a count of multiplications, a measurable bound), which is why the systems in this section work while an open-ended request to a chatbot often doesn’t, and why our invention search, which had nothing equivalent, was a much harder test.

The 2025 International Mathematical Olympiad, which ChatGPT offered in our conversation as its other main example of AI discovery, came two months after AlphaEvolve. An advanced Gemini Deep Think model solved five of six problems and scored 35 out of 42, and Google DeepMind’s announcement of 21 July 2025 says it worked “end-to-end in natural language” within the 4.5-hour time limit, with the competition’s coordinators grading its answers by the same criteria used for students. We weigh it differently from the rest, though, because Olympiad problems have answers their setters already know, so the result is evidence of reasoning ability and adds nothing new to mathematics, which is how ChatGPT used it too: as evidence that mathematical reasoning in these models is more than a disguised calculator.

OpenAI’s maths results in 2026, read carefully

The most widely shared example of 2026 is a disproof: on 20 May 2026 nine mathematicians, including Noga Alon, Tim Gowers and Melanie Matchett Wood, posted a short, human-verified version of the OpenAI-generated counterexample to Erdős’s unit distance conjecture, with their reflections on it. The AI’s argument was checked, simplified and explained by people, which is how mathematics treats any new result. Their abstract adds that the argument relies on ideas that may, at least in retrospect, be traced to earlier work by named mathematicians, which makes the result a striking case of reasoned discovery assembled from known parts, exactly what the three-kinds framework predicts a model can do. Scientific American’s report of 21 May added one further caveat: the outside experts had seen an edited version of the AI’s reasoning rather than its original output.

OpenAI’s release of 6 October 2026 was much larger: about 720 manuscripts in 372 families. Its openai/math README, which we read on 10 October, says only part of the top-line results has been formalised in Lean, a proof assistant that checks every step of a proof mechanically, and warns that some of the unformalised results could have issues. Formal checking is the mathematical equivalent of FunSearch’s evaluator, and until a result has passed it, or passed expert review, it is a claim and not yet knowledge. Our separate article on whether AI can solve unsolved maths problems looks at the release in more depth, and the short version is that a result can be surprising and correct, or surprising and wrong, with only checking able to say which.

Can AI discover new physics?

When people ask whether AI can discover new physics, they usually mean one of two very different things: a new design or method within physics we already understand, or a new law of nature. The evidence for the first is now strong, and we found none for the second coming out of a chat window. The examples below climb, roughly, from the first towards the second.

On the lowest rung sits AI-Hilbert, described by Ryan Cory-Wright and colleagues in Nature Communications in 2024. It derives formulae from axioms plus experimental data, and the authors show that famous laws, including Kepler’s law of planetary motion and the equation for the power radiated by gravitational waves, can be derived in a principled way from background theory and data, with each derivation’s validity backed by a formal certificate. That is rediscovery, and its value is as a test of the method, because recovering laws people already know is how the authors show the approach works before anyone trusts it with new ones.

The next rung is a new design that a laboratory then confirms, and the example we find most striking is a small one. Kai Wang, Xiao-Song Ma and colleagues at Nanjing University, working with Mario Krenn, reported in Physical Review Letters a way to entangle independent photons by making their origins indistinguishable, without the usual starting ingredients of photons that are already entangled or a special joint measurement of them. The paper says the specific scheme was discovered using PyTheus, an automated design tool for quantum-optics experiments. The effect was then demonstrated in the laboratory, where the photon pairs passed a Bell test, the standard statistical check that particles really are entangled. So the design was new, while the physics it used was quantum mechanics as already understood, and it was the laboratory that confirmed it worked.

The clearest large-scale design example is Urania, which the Max Planck Institute for the Science of Light announced on 14 April 2025 as a system for designing gravitational-wave detectors. The institute says the researchers found many new experimental designs that outperform the best known designs for the next generation of gravitational-wave detectors, and the underlying paper by Mario Krenn, Yehonathan Drori and Rana Adhikari appeared in Physical Review X, where the authors say the designs do so under realistic experimental constraints. Krenn’s comment on the human role deserves quoting:

“…the task of humans is to understand what the machine has done”

Krenn’s team was frank about the reversal of roles, asking what people had overlooked compared with the machine, and they have put 50 top-performing designs into a public collection they call the Detector Zoo for other researchers to study. The paper’s abstract goes further than we would, saying that analysing the best solutions uncovered “entirely new physics ideas at their core”. We read that as new ideas about how to measure, which is significant, though it stops short of a new law of nature, and the detectors still have to be built before the universe confirms them.

The highest rung we found evidence for is a model that infers forces nobody expected from laboratory data. In a paper published in PNAS in July 2025, Yu, Abdelaleem, Nemenman and Burton describe a machine-learning approach that builds in physical intuition to infer force laws in a laboratory dusty plasma, trained on three-dimensional particle trajectories, and they used it to identify large departures from common theoretical assumptions. The new physics came from a measured system, interpreted by physicists, with a model doing the inference. Without the laboratory plasma and the particle tracks recorded from it, the model would have had nothing to infer from, which is why this study marks the boundary between reasoning and evidence so clearly.

The undiscovered planet in ChatGPT’s earlier example shows where the top of the ladder sits in practice. Real planet searches rely on instruments such as HARPS on ESO’s 3.6-metre telescope at La Silla in Chile, which ESO describes as the most successful finder of low-mass exoplanets to date. A model, or an astronomer, can calculate where a planet would be detectable and what signal it would leave. Only the spectrograph can say whether the signal is there, and that measurement is the step that turns a prediction into new knowledge.

Our answer to whether AI can discover new physics, then, is that it can find new designs and methods inside known physics, and with a laboratory attached it can detect forces nobody expected, as the dusty-plasma study shows. A new law of nature needs a measurement that the existing laws fail to predict, and we have yet to see a conversation with a chatbot supply one.

Myth or truth: did AI discover new physics or create new knowledge?

Headlines say AI has solved an unsolvable maths problem, broken a 56-year-old record or discovered millions of new materials, so in October 2026 we took six widely shared examples, opened the paper or official release behind each one, and gave each a verdict: True, Partly true, Misleading or False/unsupported. The left column gives the headline as it ran and where; the right column says what the primary source actually supports.

Circulating headline or claim (where it ran)	Primary source	Verdict
“Google DeepMind used a large language model to solve an unsolved math problem” (MIT Technology Review, 14 Dec 2023)	Romera-Paredes et al., Nature, 2023	Partly true: FunSearch improved the best known cap set constructions for an open problem. The problem itself was not solved, and an automated evaluator did the checking.
AlphaEvolve “toppled a mathematical record that had stood for 56 years” (VentureBeat, Michael Nuñez, 14 May 2025)	Google DeepMind, AlphaEvolve release, 14 May 2025	Partly true: 48 multiplications for 4×4 complex-valued matrices beat Strassen’s 1969 method in that one setting. It is not a general speed-up for all matrix multiplication.
“Millions of new materials discovered with deep learning” (Google DeepMind, 29 Nov 2023)	Merchant et al., Nature, 2023; Cheetham and Seshadri, 2024	Misleading: the wording is DeepMind’s own. 2.2 million were predicted structures; 736 had been independently made when the paper appeared, and the rest remain predictions to be tested.
“AI cracks superbug problem in two days that took scientists years” (BBC News, Tom Gerken, 20 Feb 2025)	Google Research, AI co-scientist post, 19 Feb 2025; Gottweis et al., arXiv:2502.18864	Partly true: Google’s AI co-scientist proposed a hypothesis that matched an unpublished finding, but that finding had already been confirmed in the laboratory before the AI was used.
“AI Reveals Unexpected New Physics in the Fourth State of Matter” (SciTechDaily, 21 Feb 2026)	Yu, Abdelaleem, Nemenman and Burton, PNAS, July 2025	Partly true: a physics-tailored model trained on laboratory particle tracks in a dusty plasma found forces that departed from common theoretical assumptions. The experiment supplied the data.
“OpenAI announces AI’s biggest math breakthrough yet” (Scientific American, Joseph Howlett, 21 May 2026)	Alon, Gowers, Wood and six co-authors, preprint, 20 May 2026	Partly true: a real counterexample to an Erdős conjecture, verified and simplified by nine mathematicians, although outside experts saw an edited version of the AI’s reasoning rather than its original output.

All six headlines describe something that really happened and none was false, with five earning Partly true, yet every one of them dropped the checking, and the co-scientist story shows it most plainly. Google’s own post, by Juraj Gottweis and Vivek Natarajan, says a scientist supplied the research goal in plain language, and that the matching result “had been experimentally validated in the original novel laboratory experiments performed prior to use of the AI co-scientist system”. The AI reached a correct hypothesis quickly, and that is a genuine achievement. The two days in the headline, though, is the AI’s time to a hypothesis, and the laboratory work that made the hypothesis known to be correct had already been done.

GNoME earns the only Misleading label because the headline’s number does the work the evidence cannot, and the wording came from DeepMind itself: discovered implies the materials exist, when most of the 2.2 million so far exist only as computed structures. That is still a large scientific contribution, and a long list of candidates is how materials science often starts, but a business reading the headline would come away with the wrong idea of what had been achieved.

In both of the older cases the gap between claim and evidence opened within hours. MIT Technology Review’s headline said “an unsolved math problem” while its own web address said unsolvable, so the stronger wording was circulating on the day the paper appeared, and the BBC’s co-scientist report ran on 20 February 2025, the day after Google’s own post.

Can AI generate new knowledge that is actually good?

Checking headlines settles whether a particular discovery happened, which leaves the broader question of how good a model’s ideas are, and the best outside evidence on that comes from a blind study by Chenglei Si, Diyi Yang and Tatsunori Hashimoto, presented at ICLR 2025. It compared research ideas from a language model with ideas from expert researchers, and reviewers who didn’t know which was which rated the AI ideas as more novel, 5.64 against 4.84 per review, with the difference statistically significant. The same study found that only about 200 of 4,000 seed ideas per topic were unique. The comparison drew on 49 expert idea writers and 79 reviewers, who wrote 298 blind reviews between them. Rated novelty per idea and diversity across thousands of ideas are different measurements, so both findings can hold at once. Our article on whether AI can invent looks at that study more closely, along with the trial on damp in homes described below.

A follow-up from the same authors went further by having the ideas carried out. In The Ideation-Execution Gap, posted on 25 June 2025, Si, Hashimoto and Yang recruited 43 expert researchers to carry out randomly assigned ideas, each spending more than 100 hours. Each researcher documented the work in a four-page paper, and expert reviewers, again working blind, scored the finished projects on novelty, excitement, effectiveness and overall quality. The authors report that “the scores of the LLM-generated ideas decrease significantly more than expert-written ideas on all evaluation metrics”. That closed the gap seen at the ideation stage, and in the authors’ words:

“…for many metrics there is a flip in rankings where human ideas score higher than LLM ideas”

For us that is the strongest outside evidence on the question our own study tested at a smaller scale, because a novel-sounding idea and an idea that works are separate achievements and only execution or a check shows which you have. The two studies measured different things from ours, yet as the next sections show, they point the same way.

What we found when we tested it

Everything above describes other people’s systems and studies, nearly all of them built around a checker, whereas our test used the tools most businesses actually have, chat models and coding agents, with no laboratory attached. My own background matters for reading it: my working life has been websites, search and, more recently, AI applications, none of it physical, and I am not a physicist, so treat the physics in this section as reasoning to check rather than something to take on my authority.

The trolley started the search, but to test whether AI could invent at all, and not just whether it could design one product, we soon widened it, first to many everyday problems, then to a trial on one problem (finding the cause of damp in a home) and finally to industrial failures taken from public accident reports. The trolley ideas met the same end as the rest, among the 276 described below.

The tools changed along the way, from ordinary ChatGPT conversations in mid-September to Claude, through Claude Code, and OpenAI’s Codex by late September. Both are coding agents: versions of the models that can run web and patent searches, read and write files and work through long, multi-step tasks on a computer, which suits a search that has to be checked against thousands of documents. We often gave the same brief to two or more instances at once, so that one could be compared with another. In early October we designed a team of up to 25 AI agents, each a model given one role, such as generating ideas, searching for earlier patents, arguing against the others or coordinating the rest. At the end of the experiments, at our request, Codex compiled a detailed report of the whole study from the files and notes both models had left, and we quote it below where it sums up the evidence well, as the models’ own account (it is not an independent audit).

Why a patent search is a fair test of new knowledge

Patent law supplies a demanding outside check on the point we raised at the start, whether a model can go beyond what is already in its training data and on the web. The UK government’s guidance on how to apply for a patent says an invention must be new, meaning it hasn’t been made publicly available anywhere in the world, and inventive, meaning it isn’t an obvious change to something that already exists (patent law calls this the inventive step). It also has to be something that can be made and used, a technical process or a method of doing something.

That definition maps neatly on to the three kinds of new, since recombination alone usually fails the inventive test, because combining known parts in an expected way is the textbook obvious change. A reasoned discovery can pass it if the reasoning reveals something a skilled person wouldn’t have expected. And proving that an idea works in practice often needs an experiment, which is empirical discovery.

There is a second reason a patent search suits this question, which is that it forces a model to face the world’s record of what has already been done, the earlier patents, papers and products that examiners call prior art, instead of its own sense of what feels new. The same guidance notes that an application is published around 18 months after filing, and that examination can take place several years after applying, so no search, by a model or by a person, can see the newest filings, and a chatbot that says an idea appears to be novel is only reporting what it found.

Because we wanted to see what the models could do on their own, the study ran entirely at a desk under a no-spend rule, which meant AI did the patent searching, there was no private data and nothing was bought, built or tested physically.

What it took for an idea to count as patentable

The question we set was narrower and harder than whether AI is creative: could a small company, working with frontier models (the most capable models available at the time) and public research, reach an idea that met all six of the conditions below? From about two weeks into the study, each condition was tested by a desk check that needed nothing built.

1. A real, recurring problem, with a buyer who had a clear reason to pay to solve it.
2. A specific technical mechanism, as opposed to a product category.
3. A credible patent claim. That meant new (no single document or product disclosed every feature), not obvious (it survived five separate arguments of the kind a patent examiner would raise), carried by its new feature (removing that feature left half or less of the benefit) and hard to design around (no cheaper route outside the idea kept most of the benefit).
4. Physics supported by calculation: with its inputs swung 30% either way, the effect still had to be at least twice the minimum useful amount in the hardest case.
5. A cheap, private way to test the effect.
6. A believable business case.

Those rules shaped the result, and we knew it at the time: anything whose advantage could only be shown by building it was always going to fail.

The checks also explain why novelty could only ever be a hypothesis, since any search, by a model or a person, misses applications that haven’t been published yet, and a model’s search is not a legal opinion. A model can generate an idea it has never seen written down in those exact words, but it cannot infer from that fact that nobody else has thought of it, that no unpublished application exists, or that an examiner would see the combination as inventive.

New combinations were cheap

By its snapshot on 27 September our list held 276 distinct ideas across many everyday problem areas, after duplicates had been merged, and between them they cited 2,161 patent and application numbers (a count of numbers referenced across the whole list of ideas, not of patent families read for each idea). We sorted every idea into one of three groups: patentable, meaning strong enough to justify a patent application; promising but unproven, meaning not ruled out but with too little evidence either way; and ruled out. None of the 276 reached the first group, 69 ended in the second and 207 in the third, 64 of those after a first read with no search at all. Most were new in their exact wording, with no exact match in the patents and products searched, and they failed instead on whether they were inventive, whether the physics worked, how easily someone could design around them, or whether they could make money. Producing them was the fast part: by Claude’s count, AI generated 274 of the 276 in about five hours of generation, and the other two came from us, while the rest of the 25 days went on searching, checking and the later experiments. The damp trial’s 18 assessments and the 60 ideas from the October experiments described below came after that snapshot and are counted separately.

We don’t describe individual ideas here, because publishing a mechanism makes it publicly available, which under the patent rules above would destroy its novelty for anyone who later wanted to file, and 69 of them are still open questions. What we can describe is their shape. Asked to review the whole list, Codex was blunt: most ideas were a familiar product with AI added, a known mechanism moved to a new object, or software wrapped around evidence that already existed. Claude, reviewing the same list, agreed with Codex on a summary that still seems right to me: finding real problems had not been the failure, and the gap was that neither our prompts nor the models had produced a mechanism that was clearly not obvious.

Another warning came from the six structured methods we designed and ran for finding patentable ideas, which worked back from the boundaries of existing patent claims, looked for gaps no patent covered, started from problems an industry admits it hasn’t solved, carried solutions across from distant fields, ran a brainstorm built to avoid the usual answers and screened candidates for patentability first. Five of the six converged on the same obvious answer, and however different the methods looked on paper, that kind of convergence is a bad sign for inventiveness, because when several routes lead to the same idea a skilled person would probably get there too.

We then ran a trial on one problem, finding the cause of damp in a home, with Claude and OpenAI’s GPT model (running in Codex) working separately. It produced 18 assessed ideas, and 17 of the 18 had no exact match in any document the search found, yet none passed the remaining checks, since adding a feature makes an idea new as a sentence far more easily than it makes it inventive.

The same trial produced the clearest new knowledge in the whole study, and it was a reasoned discovery, though we can only claim it as new to us, the first of the three bars, since building physicists may well know it already. Working from known physics, Claude calculated that any method which only watches the water vapour in a home passively, without applying a test of its own, would stop working in the hardest case the trial allowed, so to tell the causes of damp apart in that case a method would have to impose its own controlled test. We have left out the figures, which belong to the ideas the trial assessed, so readers will have to take the calculation on trust here; it used only textbook physics, and a building physicist given the same worst case could redo and check it, which is what made it count when so little else did. Its conclusion was also negative, so it could be tested straight away against the ideas already on the table, whereas most of the 276 were positive claims that something new would work, which no calculation at a desk could fully settle. It told us something true about how the world behaves without needing a new observation: the second kind of new, working as ChatGPT said it could.

Better prompts or new data: what we tried next

By early October the list of ideas had stopped improving, so we tried changing first the prompt and then the evidence, the closest thing the study had to a controlled test of the premise I started with: if a model can only reason from what it knows, giving it something new to know should change what it produces. The agent team, built last, then gave us one more angle on the same question.

Does AI need a better prompt or new information?

We changed the prompt first, giving two separate Codex instances the same instruction: act as a future-looking inventor, treat your training data as the main constraint to overcome, and create ideas that don’t exist in it. Codex accepted it with a sensible boundary, that novelty would stay a hypothesis until a proper search supported it.

What came back looked much like everything before it, with one instance producing twelve new ideas and checking them against patents grouped by physical principle, and none was patentable. Four were promising but unproven and the rest were at or near ruled out, for reasons that read like a catalogue of familiarity: one category was established, another old, another crowded, another mature. The twelve were plausible as written and well trodden once searched, a small-scale echo of what both Si studies found when ideas that read as novel were tested harder. Asked why an explicit instruction to go beyond its training had changed nothing, Codex was blunt:

“The uncomfortable answer is that the instruction cannot force a never-documented invention to appear.”

The second instance tried a stricter prompt, banning the standard answers it kept reaching for, among them marketplaces, dashboards, memory layers and protocols. Its best result was a commercial gap with patentability unproven, so the stricter prompt narrowed the vocabulary while the underlying limit stayed where it was.

That best result also showed how generously models grade their own near-misses, because Codex gave it an in-between grade of its own that wasn’t one of our three groups, and a separate review by Claude put it back at promising but unproven commercially, with patentability unknown. Codex did keep the commercial gap separate from patent novelty, and it proposed a private test before any spend (briefs from ten potential buyers in one market, and three questions: was the result reproducible, did changing one input shift it the same way each time, and would a buyer pay?), but that test was never run.

Codex’s own recommendation shaped the rest of the study: the next step, it said, was to change the source of novelty by feeding in information from outside the model’s learned patterns, because further refinement of the prompt would keep producing clever ideas that were promising but unproven.

Experiment 1: fresh failure evidence

The first experiment changed the evidence instead of the wording, so Codex chose three industrial areas the earlier work hadn’t touched (maritime and port operations; offshore, construction and industrial equipment; and water and wastewater), all of them covered by published UK accident and incident reports, and gathered 35 recent failure reports from gov.uk sources, the Health and Safety Executive, the Marine Accident Investigation Branch’s published reports and the Environment Agency before generating a single idea. Only then did it write 30 mechanism ideas, each describing how a device would physically work instead of naming a product, without using a patent search to create them. It chose five and wrote each down in full, in no more than seven elements, before any patent searching, so the search couldn’t reshape them, and only after that did it search patents to attack them.

None of the five got through: every one was ruled out at the first check on whether it was new or obvious, so the physics and buyer checks were never reached, and the 25 ideas not chosen were never searched. Codex summed the result up in one line:

“This experiment improved the problem evidence, but it did not break the mechanism bias.”

Every promising idea had become another sensor, indicator, test rig, token (a physical tag or marker), label or recorder, and fresh evidence had made the problems sharper and more real without changing the kind of answer the model reached for.

Experiment 2: the familiar answers banned

Before the second experiment, Codex set out in writing what another failure would mean: that more ideation using language models alone was the wrong route, and that fresh access to the field, or inventing directly with the people who operate the equipment, was needed. Looking back, we think that stopping rule mattered more than either experiment, because the meaning of a second failure was fixed before anyone saw one, and without a rule like that it is always possible to argue for another prompt or another model.

The second experiment kept the same 35 failure reports and banned the families of answer the first had produced, so the main idea could not be a sensor, indicator, token, label, recorder, test rig, camera, AI classifier, RFID tag, telemetry add-on or passive witness, and every idea had to change the physics of the failure instead of recording it.

Codex produced 18 ideas that intervened in the cause of a failure and took five forward. Before searching, it noted its own suspicion that they might still be conventional adaptations of fairleads (the guides that steer a ship’s mooring and towing lines), tow connections, brakes, keyed fixtures, bypasses and valves. The suspicion was correct, and all five were ruled out: the ban had removed the earlier wording about recorders and witnesses without removing the underlying families of prior art, so the output simply moved into the next familiar family, and all ten ideas written in full across the two experiments had now been ruled out.

Codex then drew the conclusion it had fixed in advance: a language model could contribute to a patentable invention, but there was no reliable route to one from prompts, training knowledge and public web searching alone. It was careful about scope, too, saying that this did not prove no such idea existed, only that the model could find problems, propose mechanisms, structure claims and destroy weak ideas without being able to reliably originate the missing novelty by recombining public knowledge.

Codex’s recommendation had asked for information from outside the model’s reach, and that is where the experiments fell short of it, because the public records they used are indexed on the web and so were already within reach. They tested whether fresh public evidence was enough, and never tested private evidence. The four sources Codex named as the credible next step were a private maintenance, warranty or incident dataset; a domain expert describing a failure the industry accepts as unavoidable; a measured physical anomaly; and a new scientific or engineering result. None was tried. By the end of the experiments the models had suggested a dozen further routes to a patentable idea, and most of them came down to the same need for a new input, while all we had supplied was new prompts and public web data.

Can you prompt AI to think outside the box?

The agent team, the last stage of the study, came at the same question from another angle. When we set it up with Claude’s help, we wanted one agent to be an out-of-the-box thinker, unconstrained by conventional ideas or training data, and Claude declined to build it as asked, saying no agent could escape what it was trained on: “No model can.” Instead it redesigned the role so that every proposal had to bring in an outside input, such as a new fact from an observation, an experiment or an expert’s workaround, because a personality instruction to think differently would not do the job.

It also gave the job of hunting for prior art to an agent running on Codex, a different model family from the Claude agent generating ideas, so that the critic would try to break the ideas instead of agreeing with them. A generator that must cite an outside fact, checked by a critic from a different family, is the design we would start from for out-of-the-box work next time, but it remains untested: the team was used for other business tasks in early October and was never set to work on inventions.

What did ChatGPT say when we put the result to it?

Our experiment into whether AI can invent came to its conclusion on 10 October 2026, when we put the whole result to ChatGPT in the 46-minute conversation introduced at the start of this article, the same one that produced the three kinds of new. We asked, in effect, why the models had been unable to reason their way to one genuinely unique idea that could be patented, and its first explanation was that mathematics hands a system its own scoring function while invention does not, which matches what we saw in the discovery systems above.

Pushed further, ChatGPT conceded that across the study it had “repeatedly returned to established engineering concepts, repackaged as new ideas”. That sounds like support for Felin and Holweg’s backward-looking view, described earlier, but the concession needs care. It came in reply to my saying I was sure I had asked it to explore new physics, which ChatGPT accepted at once without checking, although I can find no record that I did: the nearest I came was asking for scientific thinking based on physics, maths and conservation of energy. So the concession is ChatGPT’s reading under pressure, not independent evidence, and its quick agreement is a warning in itself, since a model will often accept the framing it is given.

By the end of the conversation it had moved a long way from its opening answer, that a language model can indeed produce genuinely new ideas, and was saying that today’s frontier models “may be considerably better at solving difficult problems within established frameworks than independently discovering entirely new frameworks”. It conceded several of my objections along the way, but on one point it held its ground when I pushed: brilliant maths does not prove an equal talent for open-ended invention, and the link between them is a hypothesis worth testing. We think it was right to hold that line, and the next section explains why our own result can’t settle it either way.

What our zero does and doesn’t prove

A result of zero means that no idea reached the point where a professional search, a filing or a prototype was justified under the rules we set, and that none was shown to be new to the world. Nothing was proved unpatentable either, and the 69 promising ideas are open questions that never earned the money needed to answer them.

Some of the most useful evidence on how novelty was misjudged came from the models themselves, when Codex, asked why the instruction to go beyond its training had produced nothing patentable, set out its own process weaknesses. It had started from familiar future categories, generated products before fixing what the claim actually was and searched only after generating, so its own knowledge shaped the space it then reported on, and it had treated an idea’s absence from its memory as a sign of novelty, mistaking the edge of its own knowledge for a gap in the market. It called the result an honest zero, and did not treat that as evidence that the exercise had produced nothing useful.

Beyond the zero, the study left us with knowledge about method, starting with a traceable map of which families of prior art were already crowded. Along the way it corrected several overstatements the models had made, including the gap between counting patent numbers and counting patent families (one invention filed in several countries appears under several numbers), and it kept apart two questions that are easy to confuse: whether an idea is free to use in the UK without infringing someone else’s patent, which patent lawyers call freedom to operate, and whether it could be patented itself. It showed how unpublished applications undermine any claim of novelty, and it produced reusable ways of searching, such as looking for devices that apply a controlled test instead of only observing (the same idea the damp calculation pointed to). Those help to direct a search, though the experiments showed that even they tend to produce familiar answers unless a new outside fact changes what is possible.

The samples behind the zero are small, and the ideas were not independent of each other, so it can’t bear much weight on its own. A simple rule of thumb from statistics gives a rough upper limit on a success rate when no successes have been seen: about three divided by the number of attempts, at the usual 95% confidence. With no successes in ten tries, the true success rate could still be as high as about 30%. The 18 assessments in the damp trial allow up to about 17%, and the 276 ideas up to about 1.1%, a figure that is optimistic because closely related ideas are treated as independent. Those limits describe our process, and inventing with language models alone not succeeding in our study is a long way from showing that it always fails.

Other caveats apply:

• A single company’s study: we ran it at a desk, with no lab and no private data, and it was not a controlled trial; we chose the problems, set the checks and interpreted the results.
• Shared blind spots: the models shared training data and physical common sense, so separate attempts were not independent samples.
• Changing methods: the checks tightened during the study, so later results are not directly comparable with the first ideas.
• Desk only: working physics had to be shown by calculation, which was our design choice and says nothing about what language models can do with a laboratory.
• Unsearched ideas: 38 of the 48 mechanism ideas from the two experiments (25 from the first and 13 from the second), the ones not chosen to be written in full, were never searched at all.

The external evidence has its own limits, starting with OpenAI’s release of 6 October 2026, which was less than a week old when we published this article and which, by its own README, may include unformalised results with issues. Adam, AlphaFold, GNoME, FunSearch, AlphaEvolve and Urania are systems whose evaluators, data and laboratories did much of the work, so their success is weak evidence about a chatbot answering a prompt. The tools we used will also change quickly, and a model with a laboratory, a sensor feed or a private dataset attached is a different experiment from the one we ran, as is a model released next year.

What humans still bring to discovery

What humans can do that AI can’t, on the evidence of those 25 days, is supply the contact with the world: noticing something for the first time, measuring it, running the experiment and taking responsibility for what the result means.

The report Codex compiled at the end of the experiments described the realistic role for a model as a disciplined research partner doing five jobs:

• Mapping evidence: finding what papers, patents and products already cover.
• Proposing mechanisms: suggesting how a device might work, for a problem someone has defined.
• Translating searches: carrying a search between fields that use different vocabulary.
• Attacking weak ideas: doing it quickly and without ego.
• Keeping a log: recording what was tried and why it failed.

It was equally clear about what a model cannot do: certify that an idea is unique worldwide, supply private workshop knowledge, decide with legal authority whether an idea is inventive, or produce a non-obvious mechanism when no new technical input exists. Its verdict on the best practical use is one we would sign, a high-quality shortlist of testable hypotheses and a reliable way to reject ideas that only look patentable, which stops well short of an autonomous inventor or patent examiner.

We had tried to supply that missing knowledge ourselves, with problems I had seen first-hand, and it wasn’t enough. Claude explained why by separating two roles: the problem-holder, which was me, and the field specialist, someone who designs, builds or repairs the equipment and knows what is routine. My observations, it pointed out, had been used as topics and not as data, and nobody had recorded the measurements a specialist would want. Its examples of strong inventions paired the two roles, such as the Maclaren buggy, which combined a grandparent’s need with an aircraft engineer’s skill, and James Dyson as both user and rebuilder. It even suggested simple measurements with a tape measure, a luggage scale and phone video, none of which broke the no-spend rule. We never took them, because by then the study had widened well beyond the trolley, and in hindsight that was a mistake.

No field specialist was available to us, which was exactly why we were using AI with all the knowledge of the web. Everything Opace builds is web-based, from web design to AI applications, and in effect we were asking the web’s knowledge to stand in for an observation nobody had made, when the models could reason only from observations someone had already recorded.

Codex’s report turned this into a method for any future attempt: ask operators or tradespeople for failures they regard as unavoidable, and do it without showing them model-generated solutions, so that the person’s observation isn’t contaminated by the model’s familiar answers.

Claude’s way of putting the human advantage was that “A person can see something for the first time exactly once, and no model can do that”, which we read as meaning that a model only ever meets the world second-hand, through what someone has already written down. The history of science is full of first sightings. In our article on ChatGPT as an AI inflection point, Opace used Hans Christian Oersted’s 1820 discovery that an electric current creates a magnetic field, made while he was trying to show a class that electricity and magnetism were unrelated, and our history of OpenAI makes the same point with penicillin, found on the way to researching something else. In both cases the knowledge of the day pointed somewhere else, and the discovery began with someone seeing something unexpected and taking it seriously.

If first-hand reactions are the input models lack, the practical step is to collect them from people, and that idea came straight out of the study. After the invention experiments had ended, I built PennyForYours on 4 October as a personal hobby project, now in invite-only early access, as a way to ask real people for quick, honest reactions. It doesn’t claim that a crowd will out-invent a model, and nothing in the study suggests one would; what it gathers is human judgement and first impressions, which models cannot supply for themselves.

What would change our answer?

If we ran the study again, the design for a stronger test is already clear, and it would start by fixing the method in advance and publishing it before any work began. It would use at least three problems whose physics can be calculated and at least three model families, write the same number of ideas in full for each route, apply the same checks throughout and report how many ideas survive each check. It would also include a benchmark we like: take recently granted UK or European patents and see whether the process can rediscover their claims from a description of the need alone, stating any chance that the models saw those patents in training. We would identify the buyer and the cost of getting the decision wrong before proposing a product, fix a number of ideas in advance and change the source of evidence when they ran out instead of adding more names to the list of promising ideas (the 69 that ended up in that state show how easily that happens), and pay for a professional patent search only once one idea had a defensible, narrow claim. Even then, Codex’s report was careful to say, such a process could produce a patentable idea in principle but couldn’t guarantee one.

One result that would change the verdict below outright is a test, fixed in advance, that fed a model private field data and produced an idea that passed a professional search, a calculation and a buyer check before holding up in a prototype, because that would show empirical discovery with a language model in the loop. A chatbot making a correct prediction about the physical world that nobody had made, later confirmed by measurement, would go further still. We have seen neither yet, and we will update this page if we do.

The scaling question from earlier, whether a model with fixed weights, unlimited memory and a way to check its discoveries would keep improving, would also move the answer if anyone settled it. What this study adds is that every system in this article that kept producing results had a clean way to score them, and most real business problems don’t have one.

What should a business take from this?

The risk we worry about most reaches well beyond invention: a polished answer reads like justified belief, and teams act on it. When we compared expensive language models such as GPT-5.6 Sol and Fable 5, our conclusion was that even a premium model earns its price mainly where something outside it can verify the work, and the execution study gives that warning numbers. If an AI-generated strategy, product idea or research direction is going to cost real money, the question to ask is what has checked it, and whether that check resembled execution, which is also a fair question to put to anyone offering AI strategy consultancy on an AI-generated plan before any budget is committed. In our study that meant an attack from cited prior art, a worst-case calculation and a buyer with a reason to pay, all before anyone spent money, and in a marketing team it might mean a small live test before a campaign is rolled out.

The same logic has a commercial side for anyone who publishes. If language models mostly recombine what has been published, being one of the published sources they draw on matters, which is the logic behind generative engine optimisation, and a model can only recombine knowledge that someone, somewhere, first observed and wrote down. For a business, that means publishing first-hand material a model can’t produce for itself: your own data, tests and customer observations.

Original material of that kind has long been the core of advice from any good SEO agency, and answer engines have only raised its value, since AI SEO services can only help a business get cited for material that exists.

The verdict: can AI create new knowledge in 2026?

Our answer has three parts, one for each kind of new, and in two of them the answer is yes.

Kind of new	Can today’s AI do it?	What makes it count
Recombination	Yes, constantly, including in an ordinary chat	A reader finds the new arrangement useful
Reasoned discovery	Yes in model-plus-checker systems, and sometimes in a chat, but hard to certify	A proof checker, evaluator or calculation confirms it
Empirical discovery	Only when the AI is connected to evidence: a lab, instrument or dataset	A measurement or experiment, checked by people

We ran into the limit in the third row from the inside. Fresh public evidence and better prompts gave us sharper problems but no new mechanism, and what was missing each time was a new fact.

That is also where we would draw the line on the limits of large language models when it comes to discovery. They are strong at reasoning inside an established framework, as ChatGPT itself came round to saying, and they cannot supply the observation that a new fact needs. Given the small samples, our zero describes this study over these 25 days and should be read that way.

My opening premise was half right: you can reason well past what you know, as every new proof shows, but you cannot reason your way to a fact nobody has observed. Whether that gap is what still separates today’s AI from AGI is the bigger question our main article on AI, AGI and ASI takes further.

Frequently asked questions

Can AI have original thoughts that add to what we know?

It can produce original combinations and conclusions nobody has written down, so in that limited sense, yes. FunSearch’s cap set constructions and the Erdős counterexample of May 2026 were genuinely new to mathematics, and as far as anyone can tell the answers themselves were not in the models’ training data, although the mathematicians who checked the Erdős argument traced its ideas back to earlier work.

What it lacks is the habit of choosing when to think originally. ChatGPT made this point about itself: language models can have sophisticated reasoning abilities without reliably knowing when or how to apply them. In our study the models produced their most interesting work when the conditions forced it, with a fixed problem, a ban on familiar answers or a calculation that could be checked, and their least interesting work when simply told to be original.

We’d hesitate to call that curiosity, because an original thought that only turns up when someone has engineered the conditions for it is a different thing from a person noticing something odd and being unable to stop thinking about it.

Has AI ever discovered anything new?

Yes, and the list grows each year, but every case we could trace involved a system with a check built in, not a language model answering a question on its own. The cases covered earlier in this article run from the Robot Scientist Adam’s yeast experiments in 2009, through AlphaFold, GNoME, FunSearch and AlphaEvolve, to the PyTheus photon-entanglement scheme, the unexpected forces in a dusty plasma and the Erdős counterexample of May 2026.

Two less publicised cases show how far the pattern reaches into everyday computing. DeepMind’s announcement of FunSearch says it was also applied to bin packing, the practical problem behind loading containers and allocating compute jobs in data centres. And in June 2023 DeepMind reported that AlphaDev, a reinforcement-learning system and not a language model, had discovered faster sorting algorithms for short sequences that were then integrated into the LLVM standard C++ sort library. Sorting routines of that kind are, as the paper puts it, used trillions of times on any given day, and a new one can be checked mechanically for correctness and timed for speed, so the checker came built into the problem.

The common factor is a proposal leaving the model and meeting something that can reject it: an evaluator, a proof checker, a robot laboratory, another research team or a physical measurement. Where that check is strong, the discovery holds up, and where a headline skips it, as our myth-or-truth table shows, the claim usually shrinks once you read the paper. When we read a new claim now, the question we start with is what checked it and whether that check could have said no, and we look in the paper itself, not the headline, for three things: what exactly was new, what checked it, and whether anyone outside the team has verified it since.

Can ChatGPT discover new things on its own?

It can derive things that are new to you and, occasionally, new to everyone, particularly in mathematics. What it cannot do alone is close the loop. In our conversation ChatGPT listed the missing parts itself: recognising that an idea is new, establishing that it is correct, and adding it to what it knows. It called those separate capabilities, and said a system that combines all of them reliably remains an important unsolved challenge.

Its answer to us followed a pattern worth knowing about, because it opened with a confident yes and put the qualifications at the end, and a lot of writing on this subject does the same, so we’ve learned to read to the end before quoting the first line.

In practice, on its own usually hides help. A chatbot that searches the web, runs code or consults a memory store is already a small system, and when it claims a discovery the useful questions are which part did the checking and whether any part looked at the world as well as at text. If nothing did, we treat the discovery as a well-written hypothesis.

Is AI limited by its training data?

For facts about the world, yes, until someone gives it new evidence; for logical consequences of what it already holds, much less so. The interesting cases sit in between, and our study spent most of its experiments inside one of them, where a direct instruction to go beyond its training and a set of fresh public failure reports both left the families of idea unchanged. Asked why, Codex drew a distinction worth keeping: no idea had earned a patentable verdict yet, which is different from saying no unique idea exists.

ChatGPT made the same point from the other side by separating two meanings of going beyond the training data. If the question is whether a model’s outputs can go beyond the examples it learned from, it said yes, with substantial evidence. If the question is whether it can build an ever-expanding, reliable understanding of the universe without receiving any new external evidence, it said no, because it will eventually meet questions its existing knowledge cannot resolve.

Whether AI can discover facts without experiments depends on the kind of fact. It can discover consequences of known rules without experiments, which is a real achievement, but a fact the available evidence leaves open stays open until someone measures it, whoever or whatever does the reasoning, and that is why the discovery systems that work in the sciences, from Adam to the dusty-plasma model, all had a laboratory somewhere in the loop.

Can AI discover new physics without experiments?

Only partly. Working from known laws and data, AI can rederive formulae, as AI-Hilbert does, and propose new experimental designs, as PyTheus and Urania did, but each of those designs stays new engineering within known physics until an instrument is built and measured. Recognising a new law needs evidence that the old laws fail, and the AI result that came closest in our physics section, the unexpected forces in a dusty plasma, came from particle tracks measured in a laboratory.

What can’t ChatGPT do when it comes to discovery?

It cannot certify that an idea is new to the world. ChatGPT made part of the point itself: there isn’t even a complete, searchable database of everything people have ever invented, demonstrated or described in public. A model also struggles to search fairly for its own ideas, because the ideas and the search come from the same knowledge, and a better model would face both limits too, since they come from the structure of the problem.

It also cannot score open-ended invention the way a mathematical system scores a proof, and this was the heart of ChatGPT’s first explanation of our result. FunSearch had a clearly defined mathematical objective. Our brief, to invent something physically achievable, commercially viable, unique worldwide and inventive enough to patent, had no equivalent scoring function, and ChatGPT judged it considerably harder to evaluate than the maths it had offered as evidence of AI discovery. That is where we would put the limits of ChatGPT in discovery: at the point where nothing can mark its answer, so all you have left is how fluent it sounds.

When we use a chatbot for product or research ideas now, we try to supply the scoring function it lacks. We name the buyer, set a cost ceiling and state the worst case the idea must survive, and only then ask the model to attack its own proposal on those terms, before it gets the chance to praise it. That doesn’t turn the chatbot into an examiner, but it does mean the first answer has to survive something more demanding than our own enthusiasm.

None of this makes the models useless for discovery work, and the shortlist they can produce is valuable. Turning any item on it into knowledge still needs a person, a search professional or an experiment, and in our study nobody reached that step.

Can AI think outside the box?

It can range widely across fields, and it can follow an instruction to avoid the obvious, but it cannot leave the box of what it has been given. In our study Claude refused to build an agent that would think free of its training, and redesigned the role so that every idea had to bring in an outside fact.

For a business, the practical lesson we took was about inputs, because telling a model to be original changed little in our experiments, and banning its favourite answers only moved it to the next familiar family. What a model can’t generate is material it has never read (the complaint log nobody has published, the workaround a technician uses every week, the measurement nobody has taken), and once it is given that material its range starts to be useful. A person who has stood in a flooded pump room or watched a customer struggle with a product has material no model has read, and a model can rearrange it brilliantly once someone writes it down.

Can AI reason beyond its training data?

Yes, through reasoning and generalisation, though proving that a particular idea was truly absent from the training data is harder, as ChatGPT itself acknowledged. Reasoning beyond the data is a different matter from accumulating knowledge beyond it: a model can build on its own discoveries with enough working memory or storage and without changing its weights, but reliable checking becomes essential as discoveries pile up, and whether it can become indefinitely more intelligent using only its own reasoning remains unproven. Most arguments we read online merge these three questions and then argue past each other, when in our view the first is well supported, the second is mainly an engineering problem and the third is the open question underneath the AGI debate.

What is alien reasoning in AI?

It describes AI-produced proofs or designs that experts find unfamiliar. The Max Planck Institute said many of Urania’s detector techniques were still completely alien to its researchers, and some mathematicians have said the same of AI-written proofs. Being alien doesn’t make a result right, and strange reasoning still needs the same check as any other kind (sometimes more, because fewer people can follow it).