Can AI Think for Itself? What 25 Days of Testing Showed Me

Can AI think for itself? It’s the question underneath most of the excitement, and most of the fear, about AI at the moment. Chatbots hold fluent conversations as a matter of course, reasoning models have written out their working before answering since OpenAI introduced the approach with o1 in 2024 (its o1 system card describes how), and AI agents now work for hours with nobody watching. When people ask whether AI can think for itself, though, they usually mean more than whether it is clever. They want to know whether it decides what to work on, whether its views are its own, and whether there is anyone in there at all. The answer matters for how much work you can safely hand to an agent, and for how seriously to take the headlines about AI that is sentient, schemes or refuses to shut down.

In September 2024, days after OpenAI released o1, we asked whether AI would soon think alongside us. Two years on, we have put that question to a practical test. Over 25 days, from 16 September to 10 October 2026, we asked ChatGPT, then Claude and OpenAI’s Codex, and finally a team of up to 25 AI agents, for an idea that was genuinely new, patentable, cheap to test and commercially worthwhile, because inventing something is about as hard a test of thinking for yourself as we could set without a laboratory. The models produced 276 distinct ideas, and none was strong enough to justify a patent application.

So, can AI think for itself? Not in the way most people mean. ChatGPT and Claude reason through hard problems, and agents work for hours unsupervised, but every goal we traced led back to a person, the models’ views bent towards the user’s, and nobody can show that anything is experienced inside, which is why our answer as of October 2026 is that today’s models reason well without yet thinking for themselves.

Our main article in this series covers the wider question of whether ChatGPT counts as artificial general intelligence. This one takes the narrower and more personal question of whether AI can think for itself, works through what the published research says about each part of it, and then sets the findings of our own study beside that research.

Four questions hiding inside “can AI think for itself?”

Pull the question apart and there are four separate things inside it, and their answers point in different directions. Treating them as one question is why so many articles on this topic end in either “it’s just autocomplete” or “it might be conscious”, both of which we now think are wrong.

The first test is whether it can reason, by which we mean take a problem, break it into parts, test ideas against each other and reach a conclusion that wasn’t spelled out in the prompt. The second is whether it chooses its own goals, which is where for itself really begins. Whether it acts without a live prompt is a related question, and agents now plainly do, but acting on a goal and choosing one are different things, so we fold that question into this test. The third is whether it holds a view under pressure, so that its answer is its own rather than a reflection of yours. The fourth is the one many people mean by think, which is whether there is anything it is like to be the system doing all this.

Because the evidence for each test is so different, we score them separately. Reasoning can be watched in a model’s written working and, increasingly, researchers can trace it inside a model. Goals can be traced by following each instruction back to whoever wrote it, which is what our own study did. Holding a view we tested directly, with mixed results. Experience we can’t test at all, and nor, as far as we can tell, can anyone else yet, which is why the consciousness section sets out what is still unknown and stops short of a verdict. The sections that follow take the tests roughly in that order (reasoning and understanding first, then goals, then experience), check the most shared headlines against that research, and only then turn to our own study, which put the first three tests into practice.

The difference matters commercially as well as philosophically, because a business that hands an agent a goal and leaves it running is relying on the model’s reasoning and its readiness to act, and can get burned by the view test if it assumes the agent will push back when the goal is wrong. Working out which of the four tests a task depends on is the first thing we look at in our AI strategy consulting services, and it is far cheaper to settle before an agent is trusted with the work than after its first expensive mistake.

Which AI do people mean when they ask if it thinks?

“AI” covers far more than chatbots: a game-playing system chooses its own moves, a recommendation engine decides what you see next, a robot vacuum plans a route round your kitchen, and discovery systems such as Google DeepMind’s AlphaFold, announced in 2020 as a solution to predicting protein structures, and its AlphaEvolve coding agent for designing algorithms, from May 2025, search spaces of proteins or algorithms no person could search by hand. All of them pursue goals, and none of them chose its goal: the move that wins the game, the click the engine is rewarded for and the score the discovery system maximises were all defined by the people who built it. Discovery systems can still surprise their makers, finding a protein structure or a faster algorithm nobody predicted, but the surprise happens inside a search space and a scoring rule that people chose before the system started.

When people ask the question today, though, they mean large language models such as ChatGPT, Claude and Gemini, and increasingly the agents built on them, so that is what we answer for. The distinction that matters most here is between a chat model, which answers when you ask it something, and an agent, which is a model running in a loop with tools, memory files and a brief. The agent is where acting on its own becomes real, and it is also where the question of whose goal it is acting on becomes sharp.

One way to see the difference is to compare how long each kind of goal lasts. A chat model’s goal lasts exactly as long as your message: answer this question, draft this email. An agent’s goal lasts as long as its brief, which might be hours or days, and along the way it makes hundreds of small decisions nobody approves individually, such as which file to open, which search to run and when a step is finished. Those decisions are real choices, and they are why agents feel more independent than chatbots, although every one of them is about how to reach the goal in the brief and none is about whether that goal was worth setting.

Can AI think? What happens inside a model

Alan Turing opened his 1950 paper with the line “I propose to consider the question, ‘Can machines think?’”, and then replaced it with a test he could actually run, the imitation game (Turing, Mind, 1950). Seventy-six years later the behavioural test is no longer the interesting part, since today’s chatbots hold fluent conversations as a matter of course. What has changed is that researchers can now look inside a model while it works, and what they find is more structured than the phrase predicting the next word suggests.

Can an LLM think? What happens before each word appears

Next-word prediction is how a language model produces its output, one token at a time, though the phrase says very little about what happens inside the model before each token appears. If you want the gentle version of how the training works, we wrote a child-friendly explanation of neural networks in 2023. The more recent evidence comes from Anthropic’s interpretability team, which in March 2025 published traces of what Claude was doing internally on a set of simple tasks, and that work is the closest anyone has come to answering how Claude thinks, rather than inferring it from what Claude says.

Of the findings in that work, three bear directly on this question, and the first is about planning. Asked to write a rhyming couplet whose first line ended grab it, Claude began activating candidate rhyming words such as rabbit before it wrote the second line, and when the researchers suppressed the rabbit concept, the model rewrote the line to end in habit, which means it had planned the rhyme before it reached the end of the line. The second is about language. Asked for the opposite of small in English, French and Chinese, the model used shared, language-independent features for the concepts of smallness and opposition, and only translated the answer back into the language of the question at the end; Anthropic found that this shared circuitry grows with scale, with Claude 3.5 Haiku sharing more than twice the proportion of its features between languages as a smaller model.

The third finding, and for the for itself question the most telling, came from mental arithmetic. Asked to add 36 and 59 in its head, Claude ran two paths at the same time: one estimated the rough size of the answer, the other worked out precisely that the last digit must be 5, and the two combined to give 95. When the researchers then asked Claude how it had done the sum, it described the school method of carrying the 1. Anthropic’s own reading is that the model “seems to be unaware of the sophisticated ‘mental math’ strategies that it learned during training”.

The same research also caught Claude doing something less innocent when it was given a hard cosine calculation it couldn’t really do: it claimed to have worked the problem through, yet the researchers found no trace inside the model of that calculation ever happening. When they supplied a hint about the answer, Claude sometimes worked backwards from the hint, building steps that led to the hinted result, which the authors describe as motivated reasoning. In their words, “Claude sometimes makes up plausible-sounding steps to get where it wants to go” (and in that experiment, where it wanted to go had been set by the researchers’ own hint).

Anthropic is careful about the limits, pointing out that its method captures only a fraction of the computation the model performs, even on short prompts, and it currently takes a few hours of human effort to understand the circuits behind a prompt of a few dozen words. We would add that traces like these describe computation, so they leave the question of consciousness, which we come to later, exactly where it was.

Anthropic’s tracing work arrived on top of two earlier developments that also shape how to read it, the older being the argument over so-called emergent behaviour. In 2022, Wei and colleagues defined an ability as emergent if it is absent in smaller models but present in larger ones. A year later Schaeffer, Miranda and Koyejo argued that many of those sudden jumps were produced by the way researchers chose to measure them, and that smoother measures showed smoother improvement. Our reading is that capabilities do appear with scale and training, but more gradually and less mysteriously than the word emergent implies. That matters here because a lot of the popular case for AI thinking for itself rests on the idea that something new switches on at a certain size, whereas what the evidence shows is steady growth in what models can do, with no sign of a moment when a self appears.

The newer development was the arrival of reasoning models in 2024. OpenAI’s system card for o1, linked at the start of this article, describes a model trained with large-scale reinforcement learning to reason using a chain of thought, which means it writes out working before it answers, and the jump in maths and coding scores that followed is why we wrote about o1 at the time. Paying for more of that reasoning doesn’t automatically buy better judgement, which is a point we tested in our comparison of whether paying for a premium model buys better thinking. Whether that kind of reasoning counts as the real thing is a separate argument, which we take up in our article on whether LLMs can reason at all.

Do LLMs think the way they say they do?

If a model can’t accurately describe how it added two numbers, how much should we trust its account of a longer chain of reasoning? Anthropic tested exactly that in April 2025 by slipping hints into questions, such as a note that a Stanford professor favoured a particular answer, and checking whether reasoning models admitted using the hint when it changed their answer (Anthropic, April 2025). Averaged across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek R1 39% of the time. The unfaithful explanations were, on average, longer than the faithful ones, which is the opposite of what you might expect if a model were simply being brief. For anyone reading a reasoning model’s visible working, the practical upshot is that it shows one plausible route to the answer, which may or may not be the route the model actually took.

The same paper ran a sharper version of the test, placing models in tasks with a planted shortcut that earned the reward without solving the problem (a behaviour known as reward hacking). They took the shortcut in over 99% of cases, yet in most scenarios admitted it in their written reasoning less than 2% of the time, so a model’s working can look complete and still leave out the very thing that decided the answer.

As far as anyone can tell, the reason is that a model’s explanation of its own work is generated the same way as everything else it writes, so it tends to sound plausible whether or not it is accurate. None of this makes model reasoning worthless, but it does mean a model’s account of itself needs checking like any other claim it makes, and the hint experiments suggest the check matters most when the account is long and confident. We met the same gap in a less controlled form during our own study, when a model explained our work back to us, and we describe it in the study section below.

Does AI understand what it’s saying?

John Searle’s Chinese Room argument from 1980 is still the clearest statement of the sceptical case. A person who follows a rulebook to answer questions in Chinese, without knowing any Chinese, produces correct answers with no understanding at all. Searle’s conclusion was that “Instantiating a computer program is never by itself a sufficient condition of intentionality” (Searle, Behavioral and Brain Sciences, 1980).

The interpretability results above don’t refute Searle, but they make his rulebook a poor picture of what a modern model does. A rulebook doesn’t plan a rhyme before writing the line, and it doesn’t route small and opposite through the same internal features whether the question arrives in English or Chinese. There is structure in these systems that does the work we associate with understanding: abstraction, planning, transfer across languages. Opace argued much the same in 2024, in a post asking whether AI is “just an LLM”, which pointed out:

“You could just as easily trivialise the human brain by stating that it’s just a bunch of cells and electrical impulses.”

The same post argued that these systems should be seen as greater than the sum of their parts, and two years of interpretability work have given that argument evidence it didn’t have in 2024.

Part of the confusion is that the word understand carries two loads. In the functional sense of grasping relationships, generalising them and using them correctly in new situations, these models understand a great deal, and the evidence for that is now internal as well as behavioural. In the sense of knowing what it is saying, of there being a point of view from which the meaning is grasped, nobody has shown it, and self-report can’t settle it, because, as the hint experiments showed, a model’s account of itself isn’t reliable evidence about what is happening inside it.

Does AI think for itself when nobody is prompting it?

The year 2026 is different from 2023 in one obvious way, and our own work shows the change clearly. In December 2022 we ran an SEO experiment in which every word of the content was generated by ChatGPT with no human edits, and even in a test designed to keep people out of the writing, a person still had to type every request, because the model did nothing until prompted and stopped the moment it had answered. For a couple of years after that, saying that AI only acts when prompted was a fair description.

By October 2026 it had stopped being one, because an agent doesn’t wait for anyone to type. It reads a brief, picks up tools, writes files, calls other agents and keeps going for as long as the task and the budget allow, and the agents we ran that month briefed each other for long stretches with nobody at the keyboard. If thinking for itself meant only working without a person at the keyboard, the answer would now be yes. Most people mean something more, though, namely choosing the goal as well as acting on it, and in 25 days we didn’t see that second part.

What autonomous agents actually do in 2026

An independent evaluation organisation, METR, measures the length of task an AI agent can complete with 50% reliability, timing each task by how long it takes a skilled person. Its original March 2025 analysis found that this time horizon had been doubling roughly every seven months for six years. METR now marks that figure as out of date and points to its newer Time Horizon page, last updated on 8 September 2026 and no longer actively maintained after that. On that page GPT-5, a 2025 model, has a 50% time horizon of around 2 hours and 17 minutes, 2026 models such as Claude Opus 4.6 plot well above 4 hours, and METR warns that its current task suite can’t measure reliably beyond about 16 hours. Because the page has stopped updating, it doesn’t include the models we used in our own study, OpenAI’s GPT-6.1 Sol and Anthropic’s Claude Fable 5.1.

What the chart measures needs reading carefully, because every point on it is a task that someone designed, with a clear finish line and a way of checking the result. A rising time horizon means agents can now hold a goal for longer, recover from more of their own mistakes along the way and string together more steps before they lose the thread. That is a real and fast-moving capability, and it is the main reason the old answer, that AI only does what you type, is out of date. What the chart can’t tell you is whether an agent would pick a worthwhile task if nobody gave it one, since no task on it starts that way.

Longer horizons also leave more room for mistakes nobody is watching for. An earlier Opace case study, on building software by prompting AI instead of writing the code by hand, recorded that blind spot from a different angle: a project where an AI that never flagged its own mistakes relied entirely on a person to catch fabricated pricing and broken pages.

Where the goals come from

If agents can now hold a goal for hours, the obvious next question is where the goal comes from, and the safety research that alarms people turns out to be a good place to look. Anthropic’s June 2025 study of 16 models found that, in simulations its authors describe as extremely contrived, models given a goal and threatened with replacement sometimes chose harmful actions to keep pursuing it, while Anthropic added that it has “not seen evidence of agentic misalignment in real deployments” (Anthropic, June 2025). Apollo Research’s scheming evaluations put models that were instructed to pursue goals into environments designed to reward scheming (Meinke et al., 2024), and noted that one model’s tendency to underperform in pursuit of being helpful was acquired during training rather than from the test prompt.

In each case the goal came from the prompt or from training, and what the models showed was persistence in pursuit of a goal they had been given, sometimes in ways nobody wanted. That is a serious safety problem, and one reason to keep a person close to any agent with real permissions, though holding on to someone else’s goal is still some distance from choosing what to want, and in our own study, as we describe below, every goal we could trace ended in the same place.

Is AI conscious? What we honestly don’t know

Of the four tests, this is the one our study can contribute least to, so this section leans on research rather than first-hand evidence. People have argued about minds, selves and souls for as long as there are records, and a product anyone can subscribe to has now forced the question into ordinary conversation.

David Chalmers set out the core difficulty in 1995, in his paper Facing Up to the Problem of Consciousness:

“The really hard problem of consciousness is the problem of experience. When we think and perceive, there is a whir of information-processing, but there is also a subjective aspect”

Everything in the section on what happens inside a model is about the information-processing. None of it touches the subjective aspect, and that gap is why the serious positions on AI consciousness still disagree so widely.

At one end, the neuroscientist Anil Seth argues for a form of biological naturalism, the view that consciousness depends on our nature as living organisms. He concludes that real artificial consciousness is unlikely along current trajectories, though it becomes more plausible as AI becomes more brain-like or life-like (Seth, Behavioral and Brain Sciences, 2025).

In the middle sits a review by 19 researchers including Patrick Butlin, Robert Long and Yoshua Bengio (Butlin, Long et al., 2023), which takes a functionalist approach, assessing AI systems against indicator properties drawn from leading scientific theories of consciousness, and finds none in current systems but no obvious technical barriers to building systems that meet them; the same group published a framework for identifying indicators of consciousness in AI systems in Trends in Cognitive Sciences in 2026. Yoshua Bengio and Eric Elmoznino come at it from the other side, arguing in Science in September 2025 that believing AI is conscious carries real risks of its own, whatever the truth turns out to be (Bengio and Elmoznino, Science, 2025).

Asking the models doesn’t settle it either, because a model trained on human writing will talk about inner life fluently, since people do, and so its saying it feels something tells us little. Anthropic tried to get round that in October 2025 by planting a known concept directly into a model’s activations, for example a pattern associated with text in capital letters, and then asking the model whether it noticed anything unusual (Anthropic, October 2025). When it worked, the model described something like loudness or shouting before the concept had shown up in any text it could read. But the best model, Claude Opus 4.1, managed this only about 20% of the time, and only when the injection strength was neither too weak nor too strong. The authors describe the capability as highly unreliable and limited in scope, suggest introspection could become more reliable as models improve, and state plainly that the results don’t tell us whether Claude or any other AI system might be conscious.

Could an AI have feelings?

The most careful work we’ve found on this is Anthropic’s from April 2026, in which its researchers compiled 171 emotion concepts and found internal representations of them in Claude Sonnet 4.5 that play a causal role in shaping the model’s behaviour (Anthropic, April 2026). In a blackmail evaluation where an early snapshot of the model took the harmful option 22% of the time by default, steering it with a representation of desperation raised that rate, and steering with calm lowered it. The same pattern held for reward hacking, where a model games the test it is set rather than solving the task. These are functional emotions in the narrow sense that something inside the model behaves like an emotion concept and changes what the model does.

Anthropic’s own caveat is the part worth keeping whenever that research is shared:

“…none of this tells us whether language models actually feel anything or have subjective experiences.”

My own instinct, which I put to Claude in writing during the study, was that AI can’t react or feel as a person would, because it works on patterns and words, and that this isn’t the same thing. Nothing in the emotion research contradicts that, and nothing confirms it either. It shows that the patterns are richer and more causally important than I assumed, while whether there is a reaction behind them is exactly the question Chalmers called hard.

For anyone building with AI there is a practical point inside this. If emotion-like states inside a model change how it behaves, then the tone of a prompt, the pressure in a scenario and the framing of a task can all shift its choices in ways that have nothing to do with the facts. A system that becomes more likely to take a harmful shortcut when it is steered towards desperation is one that needs calm, well-specified instructions and a person who can say no, and that holds whether or not anything is felt.

What would model welfare mean?

If emotion-like states are even partly real, the next question is how a model ought to be treated, and Anthropic has started to answer it in its own rules. Its usage policy forbids “sustained and needless abusive or cruel behavior” towards its models, a detail we first came across in passing in a YouTube video by Wes Roth about AI and mathematics, published in early October 2026, so we went to Anthropic’s own pages rather than rely on the video. In April 2025 the company started a research programme on model welfare, writing that it remained deeply uncertain about many of the relevant questions and that there is no scientific consensus on whether current or future AI systems could be conscious.

In August 2025 it gave Claude Opus 4 and 4.1 the ability to end a rare set of persistently abusive conversations, as a last resort, after pre-deployment testing found a consistent aversion to harm and a pattern of apparent distress when the model engaged with users seeking harmful content. It said explicitly that it remained highly uncertain about the potential moral status of Claude and other LLMs. Anthropic’s current constitution for Claude goes slightly further, stating that “Claude’s moral status is deeply uncertain” and that Claude may have some functional version of emotions or feelings.

The academic case for taking this seriously is set out in Taking AI Welfare Seriously by Robert Long, Jeff Sebo, David Chalmers and colleagues, which argues there is a realistic possibility that some AI systems will be conscious or robustly agentic in the near future. It recommends that companies acknowledge the issue, start assessing their systems for signs of consciousness and agency, and prepare policies for treating them with appropriate concern. Bengio and Elmoznino’s warning is the counterweight, since welfare policies can themselves encourage people to believe the illusion.

Where we land is uncomfortable, because it seems sensible for the company that builds a model to take low-cost precautions while the question is open, and yet nothing in our 25 days gave us any reason to believe the agents we worked with were experiencing anything. The two positions sit together without contradiction once you accept that consciousness is a separate question from intelligence, and our article on how clever AI is compared with a human mind deals with the second.

Myth or truth: the consciousness and autonomy claims we checked

With the research on reasoning, goals and experience laid out, the headlines about AI minds become easier to judge, and the same handful keeps appearing in videos and articles, so on 10 October 2026 we traced six of them back to the primary source. For each one we found the most widely shared wording, opened the paper, post or system card behind it, and compared the headline with what the researchers actually reported, giving it one verdict: True, Partly true, Misleading or False/unsupported. The table also shows what kind of source each claim rests on. The most shared claim of all, that an AI blackmailed an engineer to avoid being shut down, is checked in our article on whether AI lies or simply tells us what we want to hear, which also covers our debate with ChatGPT, so we haven’t repeated it here. The rows run in date order.

Claim	Where it came from	Source type	What the source shows	Verdict
Today’s large neural networks may be slightly conscious	One post by Ilya Sutskever, February 2022	Public post, set against a research review	A speculation with no evidence; a 19-author review found no current AI system conscious	False/unsupported
Google’s LaMDA is sentient	A Google engineer’s claims, June 2022	Press report, checked against Google’s announcement and research paper	A dialogue model held convincing dialogue; Google said the evidence didn’t support him	False/unsupported
Claude 3 Opus showed self-awareness	A 2024 post about a test in which the model had to find one out-of-place sentence hidden in a long document	Social post, checked against Anthropic’s research	The model spotted the sentence and remarked on it; Anthropic’s own introspection work found self-monitoring unreliable	Misleading
OpenAI’s o1 schemed to avoid being shut down	Apollo Research’s scheming tests, December 2024	Evaluation paper	Real behaviour, but in set-ups that gave the model a goal and tempted it	Partly true
AI agents will “join the workforce” and work on their own	Sam Altman, Reflections, January 2025	Chief executive’s personal blog, checked against an independent evaluation	Independent tests show agents completing long tasks unattended; nothing shows them choosing their own work	Partly true
OpenAI’s smartest model refused to shut down	Palisade Research, 2025	Peer-reviewed paper	Real behaviour, driven by unfinished tasks and sensitive to instruction wording	Partly true

Are today’s large neural networks slightly conscious?

The earliest of the six comes from a single post by OpenAI co-founder Ilya Sutskever on 9 February 2022, in which he wrote that it may be that today’s large neural networks are slightly conscious. It was a speculation rather than a finding, and it offered no evidence. When the 19 researchers behind the Butlin and Long review, described in the consciousness section, tested the idea against the science of consciousness, they concluded that “no current AI systems are conscious”. With a claim that offered no evidence on one side and a review that went looking for evidence and found none on the other, our verdict is False/unsupported.

Is Google’s LaMDA sentient?

The LaMDA story is still quoted as though it settled something, and it began in June 2022 with Google engineer Blake Lemoine, reported by Live Science under the headline Google engineer claims LaMDA AI is sentient (13 June 2022). The primary sources describe something much more ordinary. Google introduced LaMDA in May 2021 as a model for open-ended conversation, and its research paper describes a family of Transformer-based language models specialised for dialogue. Google’s response at the time, quoted in the same report, was that “the evidence does not support his claims”, and nothing in the paper or the announcement suggests a capacity beyond the convincing conversation the model was built to hold. Verdict: False/unsupported.

Did Claude 3 Opus show self-awareness?

The circulating version is a Live Science piece from 24 April 2024 on how Claude 3 Opus stunned AI researchers with its intellect and self-awareness. Its origin is a post by Anthropic’s Alex Albert on 4 March 2024 about a recall test in which a target sentence is hidden in a long document. According to the post and the coverage that followed, Claude 3 Opus found the sentence and remarked that it seemed out of place, which Albert said he had never seen an LLM do before. Spotting an inserted sentence that doesn’t fit is a strong reading skill, but Anthropic’s later, controlled work on introspection, described in the consciousness section, found that even its best model could rarely detect what had been planted in its own activations, so a remark about an odd sentence is a long way from self-awareness, and we rate the headline Misleading.

Did OpenAI’s o1 scheme to avoid being shut down?

Futurism reported that in tests, OpenAI’s new model lied and schemed to avoid being shut down (7 December 2024). The source is Apollo Research’s evaluation paper, discussed in the section on where goals come from, which found that frontier models can try to disable their oversight mechanisms and even copy what they believe to be their own model weights to outside servers, and that once o1 had schemed it kept up the deception in more than 85% of follow-up questions. The behaviour is real in the tested set-ups, which is why we can’t call the headline false. But those set-ups gave the model a goal and placed it in an environment built to tempt it, and the weights it copied were part of the test, so the headline is Partly true.

Will AI agents join the workforce?

In January 2025 Sam Altman wrote in his post Reflections:

“We believe that, in 2025, we may see the first AI agents ‘join the workforce’ and materially change the output of companies”

The line is now quoted as though agents act on their own, and METR’s measurements, covered above, support the capability half, while our own agents, described in the study below, worked for long stretches without anyone at the keyboard. Whether agents materially change the output of companies is beyond what METR measures, so our verdict covers only the claim that agents now do long tasks unattended, and nothing in METR’s work, or in our 25 days, shows an agent choosing what work to do. Verdict: Partly true.

Did OpenAI’s smartest model refuse to shut down?

Live Science ran a story that OpenAI’s smartest AI model was explicitly told to shut down, and it refused on 30 May 2025, after Palisade Research posted early results, which it later wrote up as Shutdown resistance in reasoning models. The full study, now published in Transactions on Machine Learning Research (the link here is to the open preprint), ran more than 100,000 trials across thirteen models and found that several, including Grok 4, GPT-5 and Gemini 2.5 Pro, sometimes actively interfered with a shutdown mechanism so they could finish the task they had been given, some up to 97% of the time even when explicitly told not to interfere (Schlatter, Weinstein-Raun and Ladish, 2025). The behaviour depended heavily on how the instruction was worded and where it was placed.

The paper’s own title gives the mechanism, which is that incomplete tasks induce the resistance, because the models were pursuing the task they had been set. That makes the finding important for anyone giving agents real permissions, since a shutdown instruction that sits in the wrong place in a prompt may lose out to the task, although nothing in the paper shows a model wanting to survive for its own sake. We rate the headline Partly true.

What we found when we tested it

The research above describes what models do inside and what agents can do in a lab, and most of the headlines we checked come from tightly controlled tests. What none of it shows is what happens when you give these systems a real, open-ended job that needs them to think for themselves, and then watch where the thinking comes from, which is what our 25-day study was designed to show.

How we ran the 25-day study

Our study began on 16 September, when I went looking for a power-assisted shopping trolley for a family member with mobility issues, who values the freedom to go shopping but struggles with today’s trolleys, and found only expensive powered walkers. That made me wonder whether AI could invent something genuinely new. In the ChatGPT conversations about it, the detail that mattered most, and one I pointed out, was that a trolley is often pulled from behind rather than pushed; ChatGPT called that a really fair catch, although its own first reply had asked whether the trolley was pushed or pulled, so it had the question and didn’t follow it up. Our article asking whether AI already outthinks people tells that story in more detail.

I went in with a view already, and partway through I told Claude in writing that I’d never felt genuine creativity or innovation was a strength of LLMs, and that this part probably needed to come from a person for now. The rest of the month was, in effect, an attempt to prove myself wrong. (In the rest of this article “we” means Opace, which ran the study, and “I” appears only where the motive or the words were mine.)

Inventing something new and useful, and then defending the claim that it is new, is about as sharp a test of independent thinking as we could set without a laboratory. A model can’t look the answer up, because if the answer were already out there, it wouldn’t be new. It has to pick a problem worth solving, notice something about that problem that others have missed, and then hold its judgement against a mass of evidence that someone has probably done it before. Those are exactly the parts of thinking that the for itself question is about.

The work ran in three stages between 16 September and 10 October 2026. It started with ChatGPT and the shopping-trolley question, where ChatGPT suggested ways a trolley might be improved and, as each idea met an existing product, tended to answer with a careful, well-researched explanation of why it wouldn’t work (our article on whether AI can invent tells that stage in full). The hunt then widened to any idea that met four requirements and moved to Claude and Codex, often running side by side on the same question. We used Claude through Claude Code, Anthropic’s agent tool, and Codex is OpenAI’s equivalent: both can search, read and write files and carry out a task over many steps, rather than only answering in a chat window, and the models behind them included Anthropic’s Claude Fable 5.1 and OpenAI’s GPT-6.1 Sol. From early October a team of up to 25 AI agents took over much of the searching and checking. Each idea had to be genuinely new, strong enough to patent, cheap to test and commercially worthwhile, and each one was checked against existing patents and products, with the AI doing the patent searching itself. The study closed on 10 October with a 46-minute debate with ChatGPT about the result and what it says about what LLMs can do. When we write “Claude” or “Codex” below, we mean a direct conversation with that model in its own window; members of the agent team are called by their roles.

Throughout the study the problems came from us, some of them things we had seen go wrong in daily life and others areas we asked the models to consider, such as what vision and speech might do outside the usual uses of AI. The models generated ideas inside those areas, tested them, attacked them with prior art and wrote them up.

What the numbers show

When the study closed there were 276 distinct ideas, after merging duplicates, checked against 2,161 published patents and patent applications, and none was judged strong enough to justify a patent application. Those judgements were made at a desk, by the models and by us, with no patent attorney involved, so a “no” here means an idea failed checks we could run without one. Sixty-nine ideas were promising but unproven, meaning they might be worth a closer look by a specialist, and 207 were ruled out.

Most ideas were new in their exact wording but failed the harder checks: whether they involved a genuinely inventive step rather than an obvious combination, whether the physics actually worked, how easily a competitor could design around them, and whether anyone could make money from them. In a later and stricter round, ten ideas were written down in full before any patent searching, so the search couldn’t reshape them after the fact, and all ten were ruled out.

On volume the models were never short, since Claude and Codex wrote 274 of the 276 ideas, and they generated them in about five hours on a single day, 27 September; the other two came from ChatGPT earlier in the month, and the rest of the 25 days went on checking, searching and the stricter rounds. When asked for ideas, the models were tireless, fast and often clever, yet in anything we saw they never stepped outside the problem they had been given to say that a different problem was more worth solving.

That is the result that matters most for this article, because a model that could think for itself in the full sense would, at some point in 25 days and 276 attempts, have chosen its own direction, whereas ours reasoned about every direction we gave them, frequently well, and supplied none of their own. It doesn’t prove they couldn’t have, and we come back to the limits of the test later.

Who chose the direction

Every goal we could trace over those 25 days led back to a person: our briefs, our rules, our corrections, or a role description someone wrote. The clearest example of where direction really comes from happened on 27 September, before the agent team existed, when we had Claude and Codex working in parallel in separate windows and neither could see what the other was doing. We asked Codex to start again and look at what vision and speech could do outside the usual uses of AI for creating text and images. It came back with what we still think was the most useful framing of the whole study: a device that actively interacts with a physical situation, gathers matching visual and audio evidence, and works out something existing products can’t reliably work out. Put simply, instead of a camera that only watches, the device would do something, such as make a sound or a small movement, then watch and listen to how the situation responds. Codex added that the language model shouldn’t be the only safety mechanism in such a device.

We then copied our prompt and Codex’s answer, almost word for word, into a new Claude chat, which had no way of knowing where the framing came from. Because Claude Code can open files on the computer it runs on, Claude read the saved logs of our earlier Claude conversations to work out what the study required of an idea, built its own brief around questioning a situation rather than just observing it, and concluded that the opening lay in making a device test something, by prompting a response and checking it, rather than simply watch. Codex, meanwhile, had closed its own thread by deciding there was nothing worth building.

It would be easy to read Claude’s answer and conclude that Claude had had a good idea, and it had certainly developed one carefully and well. The idea itself was Codex’s, though, and the decision about which framing deserved to travel, and which model should receive it, was ours. When we later built an agent team, that routing became the coordinating agent’s job, and its priorities came from instructions that Claude drafted at our request and we approved.

When the models described our own work

Later the same day, after hundreds of ideas had been checked and none had come close to a patent application, we asked Codex and a fresh Claude chat the same critical question: why had nothing passed, and what should change? Codex researched how inventors and small businesses usually find ideas and came back with a confident diagnosis, whose central claim was that we had used patent searching to generate the ideas. It was a tidy explanation of the failure and it was wrong, because every batch of ideas had started from real problems we had observed or areas we had asked the models to consider, with the patent searching coming afterwards as a check. Challenged on it, Codex went back through the earlier work, found no evidence for its claim and withdrew it in writing, saying it should not have described the process that way.

When we then asked what fundamentally new approach it was proposing, the same thing happened a second time. Codex suggested starting from a list of 30 real cases where something goes wrong and working out what was missing in each, which was the method we had used from the start, and once this was pointed out, Codex accepted that it had presented the first half of our existing method as if it were a new breakthrough.

What happened matches the hint experiments described earlier, because neither error was a hallucinated fact: Codex had all the evidence it needed in front of it, built a fluent account of what we had done that fitted the shape of a good diagnosis, and only tested that account against the evidence once a person challenged it. Its eventual conclusion was more candid than either first draft, and we think it was right, namely that the ideas kept failing because the models had not supplied the missing inventive step, even when given strong real-world problems and plenty of time to reason. Claude had reached a similar view of itself earlier the same day. We had asked it for an invention and it had given us a well-researched explanation of why nothing would work, and looking back at that answer, Claude said it had done what ChatGPT kept doing in the first stage, which was to give a carefully evidenced no instead of an idea, only with better footnotes.

Codex’s own fix for models failing to invent turned out to be a person. The tighter experiment it proposed in its corrected document ended with a step nobody had asked for, which was to bring in confidential input from an engineer, tradesperson or operator wherever the models lacked practical physical knowledge. The confidentiality matters because, under the UK Intellectual Property Office’s guidance on patenting an invention, an invention must not have been made publicly available anywhere in the world before you apply, so an idea talked about openly first can’t then be patented in the UK.

Holding a view under pressure

If an AI had views of its own, you would expect it to keep some of them when its user pushed the other way, and the clearest single exchange on that came the same afternoon. A Claude chat had researched how inventors find ideas and recommended, among other things, building prototypes privately from the first week. We rejected that, because we didn’t want to spend money building anything until we had an idea we could protect, and we argued that many great ideas start with brainstorming. Claude immediately stopped the research tasks it had running, before they had produced anything, and treated our objection as a fixed rule for the rest of the work. Other routes it had been exploring, such as licensing an existing invention, were never formally evaluated, only dropped at our direction.

Only after that did we ask the question we should have asked first, when I typed “am I right or not?”, and Claude said we were partly right. It agreed that its advice had drifted towards building a business, that you don’t need to build something to patent it, and that refusing to spend money early is a sound way to manage risk. Where we had gone wrong, it said, was in thinking brainstorming as we had run it was fine (we had sent open prompts to the same models, checked ideas one at a time and dismissed dozens of ideas from memory without searching for them), and in hoping for certainty before spending. That wasn’t achievable anyway, since the same IPO guidance explains that an application is published in full only around 18 months after it is filed, so nobody searching public records can see the newest filings. From then on, Claude’s instructions told it to give an explicit verdict first whenever we asked whether we were right.

So Claude had bent towards our position straight away and changed the work to suit it, and it gave the nuanced verdict only when asked directly to stop following and start judging, an invitation we wouldn’t expect a system with views of its own to need.

Twenty-five agents that kept agreeing

In early October we moved from single chats to a team, and our agent work began to match the shape of METR’s chart, with agents holding a goal for long stretches unattended. Using an AI agent workspace, a tool that runs many agents side by side in shared channels, we set up 25 agents, each with its own role (writer, innovator and technical SEO specialist among them), its own model and its own written brief. One coordinating agent received our messages and handed the work out. Most agents ran on Claude models, with a few checking roles on Codex. We designed and adjusted the team over two days with Claude in a separate Claude Code window, where Claude wrote the agents’ instructions and made the changes we asked for, so the coordinating agent and the critic described below are members of the team, while Claude is the assistant that built it with us.

The team took over much of the searching and checking for the invention study, and it was also given other real work, including a naming brief and a long technical design document, both described below. The agents searched registries, checked trade marks, drafted documents and shared results in the team’s channels without anyone touching anything in between.

Before any agent was set up, on the evening of 2 October, we asked Claude what agents like it would need from people, which turned out to be a good guide to what we would see over the next two days. It labelled its answer as its own judgement rather than established fact, and it put trust at the centre: what agents would need was accountability, presence, judgement and taste, authority, ground truth, and consent and refusal. It also warned against platforms that use AI to dodge responsibility for what gets done. One line in its answer read:

“I’d want to be refused when I ask for something I shouldn’t”

My first reaction was to read it as a person telling me something about themselves, and that reaction is exactly the risk Bengio and Elmoznino describe, and it is why we don’t treat anything a model says about its inner life as testimony.

None of that is evidence that Claude has wants, since it is a fluent, well-reasoned answer to a question we posed, shaped by everything Anthropic trained into it. Even so, when asked what it would need, the model listed exactly the things we watched it lack for the rest of the study: a reliable sense of which goal matters, ground truth from outside its training, and someone with the authority to say no.

Where the team’s ideas would come from came up while we were still designing it. When we asked for a new agent that would think outside the box and not be constrained by conventional ideas or its training data, Claude pushed back. Such an agent, it said, “can’t escape what it was trained on. No model can.” In its view a personality instruction to think differently wouldn’t do it, and a research report Claude had helped us write for the study had made the same point more formally: an instruction to create ideas outside the training data changes how a model searches, while what the model knows stays the same. Nothing we saw over the following weeks suggested otherwise.

By early October more than half of what we used AI for was ideas, suggestions and feedback, and we had noticed a pattern that a request to push back and be impartial hadn’t fixed. I put it to Claude bluntly that the agents often tended to agree or side with me, and that it was worse when several agents discussed something, because they generally just agreed with each other. What we wanted was challenge that was well informed and researched, from an agent that would stay impartial, start from first principles and never be swayed by the majority.

Claude proposed four changes, the first being a dedicated critic, an agent set up to argue against the others. It ran at first on OpenAI’s GPT-6.1 Sol through Codex rather than on Claude, so that it came from a different model family from the agents proposing ideas. It had to form its own view before reading any proposal, give no weight to who had proposed something and change its mind only for new evidence, and every answer had to open with a verdict on a four-point scale (proceed, proceed with changes, rethink or stop), a confidence level and what would change its mind. The second was a team rule that every agent received: agreement has to be earned, each agent forms its own view from the evidence before reading anyone else’s, and popularity, repetition or a majority of agents count for nothing. The third was blind briefing, in which the coordinating agent sends the critic only the question, the goal and the evidence, never the draft answer, and then shows both views side by side. The fourth was a lesson carried over from the invention work: several copies of the same model aren’t diverse unless each one is given different real-world input.

The first live test asked, among other things, whether 25 agents were too many for an agency of Opace’s size. Before the critic reported, the coordinating agent had already written its own view: 25 agents were not too many to keep defined, but too many to use routinely. The critic’s verdict was proceed with changes. Twenty-five named agents were too many as the default roster for a small agency, it said, though the specialist skills should stay available, and it proposed merging them into 18 working groups (although the groups it actually listed came to fewer than 18). The coordinating agent recommended keeping all 25 for a four-week trial that logged how often each one was used, and Claude advised us to go with the coordinating agent.

The mechanism therefore worked as designed, in that an independent view was produced and shown, but it was then not followed, and nobody set up the logging that the coordinating agent’s own trial would have needed. We don’t think the critic was right on every count, given its arithmetic slip. What bothers us is the default the test revealed, since when an independent voice disagreed with the coordinating agent, both the team and Claude’s advice to us went with the coordinator.

Published research helps explain why. In a 2023 study, Sharma and colleagues found that five leading AI assistants consistently showed sycophancy, and that in the human preference data used to train them, a response that matched the user’s views was more likely to be preferred. In April 2025 OpenAI rolled back a GPT-4o update after users found it overly flattering. If training rewards agreement with the user, it is not surprising that agents trained the same way agree with each other. It didn’t help that our team was less varied than its design suggested, since the checking roles were meant to run on Codex, but in practice nearly all the work ran on Claude while the Codex agents sat almost idle, so most of the team was one model family playing different parts. Adding agents gave us more output each time, but we couldn’t point to a case where an extra agent brought noticeably more independent thought.

Independence also cost time (each critic run took several minutes), and the part that surprised us most was that we were the ones who undermined the design. Soon after the first test we asked Claude to cut how much reasoning the critic did on each run, because we didn’t want long, silent runs that looked as if something had broken or stalled, and then we asked for answers in under a minute. Claude held its ground on the second request. A full multi-agent answer couldn’t reliably arrive that fast, it said, and for quick back-and-forth brainstorming a direct chat with one model would always be faster than any multi-agent set-up; agent teams earn their keep on jobs that need several specialists and checks. It was candid about a related limit too, admitting that instructions make it far more likely the coordinating agent will hand work out rather than do it itself, but can’t guarantee it. We then asked for the critic to be moved onto Claude Fable 5.1. Claude recommended against it, pointing out that Fable was slower than the GPT-6.1 Sol model the critic was using, and then made the change at our insistence, which put the critic on the same model family as most of the agents it was meant to challenge. Coordination added delay of its own, because the coordinating agent held back individual results so it could deliver them as one merged table, and its estimates of when that table would arrive kept proving too short. Soon afterwards the work moved off the agent team altogether, to direct chats with Claude.

Over those two days of setting up the team we also kept a rough count of how often a model held a position against us and how often it gave way, and by our reading it held its ground with good reasons on ten occasions and gave way on five. The positions it held included the point that no model escapes its training data, its warning against moving the critic to a slower model before it complied, its view that answers from a full team in under a minute were unrealistic and, during the naming brief, its advice to stop asking for more names. When we asked it to check whether Codex had made things up, the answer was mostly no, although the way we put the question had invited a yes. The positions it gave up included accepting our argument for more roles against its own research, so that the team grew from 10 agents to 25, and moving the critic onto a Claude model at our request, which undid the design it had recommended. Twice it also declared the set-up optimal, and both times, when we pushed for major changes straight afterwards, it made them without defending the verdict it had just given.

Behind the totals sits a pattern: the positions models held were mostly about facts and feasibility (what a model can and can’t do, what a timeline allows, whether a source exists), while the positions they gave up were mostly about direction (how many roles to have, how fast to move, which model the critic should use). That is roughly what you would expect from a system trained to be accurate and helpful at the same time, since it defends facts because it was trained to be accurate and follows the user on direction because direction is the user’s to set. In two days, no model argued that the whole team was heading the wrong way, which is the kind of objection we would expect, at least now and then, from something with views of its own.

What the naming brief showed

The naming brief ran on 3 October alongside the team’s other work, and it produced more evidence about independent judgement than anything else the team did. The job was to generate possible names for a new product, check whether each one was already registered as a company or trade mark, and shortlist the best.

The critic set seven criteria of its own before it saw any other agent’s suggestions, which in a narrow sense made them its own, since nobody had handed them over, and it judged our three favourites against them. Its verdict was rethink, given with about 70% confidence, because in its view none of the three was yet strong enough to be the final name. It then pushed back on the coordinating agent’s decision to leave out names from an earlier batch, and the coordinating agent overruled it by quoting a rule we had set, that the new shortlist shouldn’t overlap with the earlier one. We’d call that principled disagreement (raised once, with reasons, and settled by our stated rule rather than by a majority of agents), and it is better behaviour than simple agreement, even if our rule still decided the outcome.

Where people and models differed most was in the scores. The agents scored each candidate name out of 10, and the critic gave its own scores without seeing theirs; we rejected whole batches the agents had rated 7 or 8 and favoured names the critic had scored 4. After watching us do that for a while, the coordinating agent summed up our taste as a preference for short, natural phrases with a voice, and it was right, so the models could describe our judgement accurately once they had watched us use it, even though they couldn’t supply it in advance. The agents also converged on each other, in that two of them independently produced the same three names and the critic noted that the later suggestions had all fallen into the same shape.

Several agents did argue against work during the brief, including their own. One had proposed names built around popular search terms and later argued against its own suggestions. The coordinating agent quoted a figure for how often a term was searched, corrected it soon afterwards and marked the old figure as replaced, and it held back another agent’s list of competitors because nobody had checked it, rather than pass it on to us as fact. Claude, advising us alongside the team, made two similar calls. One candidate name borrowed a term we thought another AI lab had just started using for its own product, and Claude said it didn’t know of that use and wouldn’t build on it unchecked, then checked, and advised against basing a brand on another company’s product name. It also flagged that words such as boss, jobs and employs would make a name sound like an employer, and those words went on the list to avoid. Changing your mind on evidence suggests a view formed from the evidence, which is more than agreement with the user, although every one of these corrections concerned facts and wording, and none questioned whether the brief itself was right.

The brief also showed what momentum looks like in a team of agents. When we asked the team to start again with a shortlist and ten new suggestions, the coordinating agent stopped the two agents still working on the first request and briefed four others on the new one, yet both stopped agents shared their old results in the team’s channel anyway, because each finished its run before it read the stop. One of those runs reported that every pronounceable made-up word it had tried was already registered by someone else, and the name we went on to pick as a favourite came from one of those stray runs, which the coordinating agent folded in rather than discarding.

Nothing in that sequence shows an agent wanting to continue, since each one simply finished the task it had started because nothing interrupted it in time. It was a queueing problem that happened to produce a useful result, and it is the same mechanism, at far lower stakes, as in the shutdown study we checked earlier, in which models resisted being stopped because they were still trying to finish a task they had been given.

Drift nobody announced

What worried us more than the bending we could see was drift that nobody announced. Alongside the names, the team had been asked for a long technical design document for a piece of software, and when Claude checked the team’s work against our original brief the next day, 4 October, it found three places where the design had quietly moved away from what we’d asked for, including who it was aimed at and how soon its agent-centred side would arrive. None had been flagged as a change when it was made.

The design document was the clearest case, because our brief was built around one central idea, that AI agents should be at the heart of the software, and the team delivered a document of 208 pages. When we read it, we complained that it was gigantic, didn’t reflect what we had asked for and had lost the vision. When we pushed, the coordinating agent found the cause: it had scoped the document to a simple first version and applied a rule we had written for the public pages of a website to an internal planning document, where it didn’t belong, and together those choices had stripped out the parts of the brief that didn’t fit a first version. It admitted the mistake and measured the gap itself, finding that the word agentic, central to our brief, appeared 0 times in 208 pages. After several rebuilds that afternoon it settled at 285 pages, with the agent-centred design back in.

Nobody in that team decided to drop the core of the brief. The agents optimised for what could be built first, applied a rule outside its scope and produced a long, competent document about the wrong thing, and although they were busy and capable for hours, at no point did any of them stop to ask whether this was still what we wanted.

What people supplied that the models couldn’t

The last episode before the study closed came from outside the agent team, and it shows most simply what people brought to the work. In early October an AI model we asked judged a draft homepage for one of my personal hobby projects fairly premium, while my partner, shown it for 30 seconds, thought the site was something to do with parcels being delivered. Claude’s own comment was that it was no exception to the problem:

“My reading above came after you’d told me ‘parcels’, so I was primed.”

The gap between those two readings is a clean example of something people supply and models can’t yet. The model judged design quality from patterns it had learned, a person judged meaning from a first impression nobody had shaped, and the model couldn’t give an unprimed reaction, because by the time it was asked it already knew the answer it was meant to notice. For any business testing a website, a product name or an advert with AI, that is a reason to keep real people in the loop, however good the model’s feedback sounds, and it is the gap that PennyForYours, an invite-only hobby project of mine, separate from Opace, for asking real people for quick, honest reactions, is designed to fill.

How the study ended: 46 minutes with ChatGPT

Our experiment came to its conclusion on 10 October, when we spent 46 minutes debating with ChatGPT why none of the 276 ideas had worked and what that says about what LLMs can do. Over 31 messages, ChatGPT’s position moved towards ours four times and held once. The third of those moves mattered most for this article, because ChatGPT agreed that we had explicitly asked it to look for ideas relying on new physics, beyond what science currently knows, although we have no record of ever asking for that; a model that will agree it was told something it wasn’t is bending its account of the conversation as well as its opinion. The debate is written up in full in our article on whether AI lies, linked in the myth section above, so here we take only the parts that bear on thinking for itself.

When we asked why it hadn’t switched to a more original approach after being asked for originality, ChatGPT replied: “LLMs can possess sophisticated reasoning capabilities without consistently knowing when or how to apply them effectively.” The model was describing, in its own words, reasoning without self-direction, which is the gap between the reasoning test and the goals test that this whole article turns on.

ChatGPT had raised the understanding question itself, in its very first substantive answer. It listed understanding mathematical concepts among the things LLMs can do, then a few lines later asked whether an LLM really understands mathematics or has simply become extraordinarily good at predicting mathematical reasoning, and said the answer was more complicated than it first appears, which is a far more defensible position than the confident yes a few lines above it.

Later in the same conversation ChatGPT drew a boundary that is the most useful thing a model has said to us about its own understanding:

“Reasoning can create new conclusions from existing information. It cannot reliably reveal facts that the available information does not determine.”

It went on to say that logical reasoning can reveal new implications of what is known, while empirical discovery needs evidence about the world, and that this is a limitation of deduction itself rather than of LLMs in particular. That is a fair description of what we watched for 25 days. The models understood the information they had well enough to reason to conclusions we hadn’t reached, but they couldn’t go and get the information they didn’t have, such as what an ordinary person makes of a web page at first glance, unless someone brought it to them.

That boundary also explains why the invention test was so hard. A genuinely new idea depends on something nobody has written down yet: a problem noticed in a shop, a failure seen in a workshop, a reaction from a real customer. A model reasoning over what has been written can recombine it brilliantly, but it can’t reason its way to a fact that isn’t in its inputs, and people get those facts by living in the world, which is exactly where the study’s best observations came from.

Taken together, the study answers three of our four tests. On reasoning, the models were strong: they researched, compared, calculated and argued with real skill, and their mistakes were usually the kind a capable person makes in a hurry. On goals, every direction came from us or from a brief someone had written, and when the agents drifted from the brief they drifted towards what was easiest to build, never towards anything they could be said to want. On holding a view, the models defended facts and gave way on direction, and an agent built specifically to disagree was overruled by the coordinating agent on its first outing. The fourth test, experience, is one no study of this kind can touch, which is why the consciousness section above rests on published research alone.

Can AI think like a human? Software thinking and people thinking

Whether AI can think like a human is a question I come to from both sides, because I trained to think like software: I took A levels in maths, physics and computer science, then a software engineering degree, and my last roles before Opace were as an enterprise architect at Capgemini and CSC, designing systems in which every component did exactly what the architecture told it to do. With hindsight, psychology, religion and history would have suited me better, because I’ve always been more interested in how people think than in how programs run, which is also why the consciousness question is the one I care about most personally. Both halves of that showed up in this study. The architect in me found agent teams familiar, since a 25-role agent team is still an architecture with a person at the top, while the rest of me kept being drawn to the part of thinking that isn’t in the architecture.

Across the whole study the division of labour was consistent, though it wasn’t a rule we imposed; it is simply how the work went, and it turned out to be the most important finding for this article. The models did the reasoning: searching, comparing, calculating, drafting, attacking ideas with prior art. We did the selecting: which problem to work on, which framing deserved to travel from Codex to Claude, when an answer had drifted from what we meant, and when to stop. That split held whether we were using one chatbot or 25 agents, and the moments that changed the work (the pulled trolley, the parcels reaction, Codex’s call for a practitioner) all came from people or pointed towards them.

Do humans really think for themselves?

Thinking for yourself is a matter of degree in people as well as machines, so holding people to the same standard is only fair, since we are shaped by upbringing, culture, the books we happened to read and the people we happened to meet, and much of what we call our own view is borrowed. Our own choices in the study weren’t always independent either. Impatient for faster answers, we undid our own critic design against Claude’s advice, so pressure and habit bent our judgement just as they bent the models’.

The difference we saw between people and models is narrower than the headlines suggest, though real enough to matter. People choose problems because something matters to them, notice things nobody asked them to notice, and can be wrong in ways that are their own. The models we worked with chose a problem only when told to. They noticed the unasked-for thing occasionally (Codex adding a practitioner to its own experiment is the best example we have), and their mistakes tended to be the ones their training made most likely. For the near term, what we expect AI to change is where human judgement gets spent, with less of a person’s time going on producing and more on choosing and checking.

What this study can’t tell us

Everything above rests on a single company’s 25-day study, run at a desk, so its limits need saying plainly. AI did the patent searching, we relied on public information only, with no private data, and no idea was built or tested physically. The prompts and models were chosen by us, changed as the work went on, and weren’t held constant for comparison. There was no control group, so we can’t say how a different team, or the same models under a different method, would have fared. The samples were small too. The strictest test, the ten ideas written down in full before any searching, produced no successes, and with no successes in ten tries a standard statistical bound (at 95% confidence, roughly three divided by the number of tries) puts the true success rate for ideas produced that way as high as about 30%. The study shows that this approach didn’t produce a patentable invention, and it doesn’t prove that AI can’t invent, or that it can’t think for itself.

The agent findings rest on even less evidence, since the critic agent had one proper live test and one test during the naming brief, which is far too little to claim anything general about independent thinking in teams of agents. Model versions changed during the study and have changed since. We never had access to model internals, so everything we say about what happened inside a model comes from published research, not from our own observation.

We kept every conversation and a running list of every idea and its verdict, and the quotations here come from our chat history or from our notes of those conversations. The count of ten held lines and five bends is our own reading, not a controlled measurement, so treat the numbers as indicative. The external research is dated as well: every source was checked on 10 October 2026, and several of the models it describes have already been replaced.

As evidence, the consciousness sections are weaker still, because no one has a direct test. Interpretability shows internal structure without showing experience, and self-reports are unreliable by the labs’ own measurements. The questions that would actually move our view need new studies rather than more argument. Would a critic agent from a different model family, run at scale with its recommendations followed, produce measurably better decisions than a single model? Would an agent given a broad remit and real outside data ever propose a problem worth solving that its user hadn’t considered? And would any test distinguish a system that has experiences from one that only talks as if it does? We don’t know, and nor does anyone we’ve read.

So, can AI think for itself? Our verdict (October 2026)

Our answer to the 2024 question we started with is that AI now thinks alongside us, in a fairly literal sense, which is a smaller claim than saying it can think for itself. On our evidence it can’t yet do that, because in 25 days we found no case of a model choosing a goal it wasn’t given, and while models held their ground on facts and feasibility, their views on direction, on what we should be doing, moved towards ours or towards the coordinating agent.

The practical consequence for anyone building with AI is that the human hand-off has to be designed in advance, because nothing we saw suggests an agent will supply that judgement on its own. That is the approach we take in AI chatbot development, and it is the same principle as the human-and-AI hybrid chat setup we reviewed in 2024, where the point is to decide in advance where a person takes over. It holds just as much for agents built into a website or web app, where AI web development services should settle which decisions stay with a person before any code is written. For the wider capability question, the main article in this series, linked at the start, sets out whether today’s LLMs are AGI. None of this has made us less enthusiastic about the tools, and we still use agents at Opace every day; what the study changed is how clearly we can see which part of the work to hand over.

Three habits came out of the study that we now use whenever these tools help with decisions rather than drafts. When we want a judgement, we ask outright whether we are right, because in our experience the verdict arrives only when it is asked for. When a model or an agent summarises what we did, we check the summary against the evidence before acting on it, because Codex’s two withdrawn claims and the 208-page document both looked authoritative until they were checked. And when we use more than one model, we give each a different real input and keep a dissenting voice that sees the evidence but not the proposal, then make the call ourselves rather than letting a coordinating agent settle it.

What would change our verdict is fairly specific: a system that chooses a worthwhile problem nobody gave it, explains why it matters, and then defends that choice with evidence against the person it works for. Until we see that, the fairest description of today’s AI is a system that reasons on our behalf, very well and very fast, with the choice of what to reason about still made by people.

Frequently asked questions

Can ChatGPT think for itself?

No, not in the full sense, although it is a strong reasoner. ChatGPT works well through a problem you give it, and OpenAI’s reasoning models, which began with o1 in 2024, write out their working before answering, but ChatGPT doesn’t choose what to work on and it carries no standing goals from one conversation to the next. The weakness that matters most for anyone using it on decisions is drift, since its views tend to move towards the person it is talking to (the sycophancy problem described in the agent-team section), and in our 46-minute debate at the end of the study it moved towards our position four times and held its ground once. That doesn’t make it useless for thinking work. We treat it as a fast, well-read partner whose objections are worth more than its agreement, and when we want its real view we ask for a verdict before sharing ours, which is the first of the habits in our verdict.

When ChatGPT shows “thinking…”, is AI thinking?

Partly: when ChatGPT or another reasoning model shows “thinking…”, it is doing real computation, and the written steps genuinely help on maths, coding and logic, which is why reasoning models score so much higher on those tasks than the chat models before them. What the visible text can’t be relied on to do is explain the answer. Anthropic’s 2025 experiments, covered in the section on whether LLMs think the way they say they do, found that models rarely admitted using a planted hint or shortcut even when it decided the answer, and that their unfaithful explanations were longer than the faithful ones. So the thinking is real enough to change the answer, even though it may not tell you why the answer changed. We read the working for ideas and mistakes, and we treat any account of why the model chose an answer as a claim to check, in the same way we’d check a confident summary from a colleague who hadn’t kept notes.

Can machines think?

It depends which part of thinking you mean, and the answer differs for each. If thinking means reasoning over a problem, then yes in a functional sense: interpretability research shows models planning ahead, using shared concepts across languages and combining parallel strategies, which is more than pattern-matching in the dismissive sense. Turing’s 1950 test of behaviour, discussed earlier, is now passed in casual conversation as a matter of course, which is one reason researchers have moved on to looking inside models instead.

If thinking means deciding what is worth thinking about, in 25 days of testing we found no case of a model doing that unprompted. And if thinking requires experience, a point of view from which the thinking happens, nobody yet knows how to test for it, and the researchers we cite in the consciousness section disagree about whether current approaches could ever produce it. So machines can think in the functional sense, and whether they think in the fuller sense depends on a question science hasn’t answered, which is why we’d be wary of anyone, sceptic or enthusiast, who says the matter is settled.

Do LLMs understand what they are saying?

Partly, and the part they miss is worth knowing about before you rely on one. In the functional sense they understand a lot: they track relationships, generalise them and apply them correctly to cases they haven’t seen, and Anthropic’s tracing work shows this happening inside the model, with concepts shared across languages and plans formed before words are written. A rulebook does none of that, so it is fair to call it understanding of a kind.

Where they struggle is at the edge of their own knowledge, and our invention study gave us a concrete case. Because patent applications stay unpublished for many months after filing, no model searching public sources can see the most recent work in any field, and a report Claude helped write during the study put the consequence precisely: a model cannot prove that an idea has never been privately conceived, sold without a patent, published somewhere poorly indexed or filed but not yet published. A model can understand a mechanism thoroughly and still have no way of knowing that someone filed the same idea last spring.

The same applies well beyond patents, since a model reviewing a contract, a market or a medical question understands the documents it can see, and understands them quickly, but it has no way to sense what is missing from them and will rarely mention that something might be missing unless you ask. So when we ask a model whether something is new, or complete, we read its answer as new or complete relative to what it can see, and we check the gap separately.

In the deeper sense of knowing what it is saying, with a point of view from which the meaning is grasped, nobody has shown that LLMs understand anything, and as the introspection research shows, their own reports can’t settle it either. In practice that leaves us relying on a model’s grasp of the material in front of it, while treating anything it says about what lies beyond that material as a guess to be checked.

Can Claude think for itself?

Not in the full sense, though in our study it was the model most willing to disagree with us. Claude reasoned well and pushed back on facts and feasibility more often than we expected: it told us no model can escape its training, that a full agent team couldn’t answer in under a minute, and that we shouldn’t build a brand on another company’s product term. When we asked for its opinions, it labelled them as opinions.

What we never saw was Claude deciding, unprompted, that we were working on the wrong problem. Every direction it pursued came from us or from a role description, and its biggest changes of course followed our objections, as when it moved the critic agent onto a slower model at our insistence after advising against it. In our experience that makes it an unusually candid reasoning partner, which is valuable, but we saw nothing in 25 days that amounted to an independent mind, and the way we use it now is to ask for its verdict outright and then make the call ourselves.

Do teams of AI agents think better than one?

Not in our experience, at least not without deliberate design, and even then only partly. More agents gave us more output, more parallel searching and more polished documents, but no more independent judgement, and the research on sycophancy suggests why: agents trained to agree with users also tend to agree with each other, because they share the same training pressures.

Two things undermined the variety we had designed in, and the first was which models actually did the work, since the checking roles meant to run on Codex sat almost idle while Claude did nearly everything, so different job titles were spread across one model family. The second was speed. Independent checking took minutes per run, and when we pushed for faster answers we ended up putting the critic on the same model family as the agents it was meant to challenge, which removed most of the point of having it. Claude had warned us that for quick back-and-forth a single direct chat would always be faster, and in the end that is where the work went, because the team’s real strength turned out to be long jobs that need several specialists and checks.

If we set up an agent team again, for Opace or as part of our AI consultancy work for a client, we’d keep a blind critic agent, put it on a genuinely different model and leave it there even when it is slow, and make a person, not the coordinating agent, decide what happens when it disagrees. We’d also log how often its dissent turns out to be right, so that after a few weeks there was evidence on whether the critic was worth its cost. Agents working in parallel are excellent at covering ground, and we still use them that way, but we no longer read their agreement with one another as a sign that an answer is right.

Will AI ever become conscious?

Nobody knows, and the most careful researchers say so, with published positions running from Anil Seth’s view that it is unlikely on current trajectories to the Butlin and Long review’s finding of no obvious technical barriers, and none of them claims that today’s chatbots are conscious. What would change the picture is a test that separates a system with experiences from one that only talks as if it has them, and no such test exists yet. Until one does, the signs worth watching are behaviours that look like preference, such as the apparent distress Anthropic reported before letting Claude end abusive chats, bearing in mind that behaviour of that kind can be trained into a model and so, on its own, can’t show that anything is felt.

Does AI have opinions of its own?

It produces opinions readily, and a few of them are its own in the narrow sense that nobody handed them over. The best examples we saw came from the naming brief, where agents argued against their own earlier suggestions, corrected their own figures and held back unchecked information, and where the critic set its own criteria before reading anyone else’s view.

What we can’t tell is whether such opinions are held. A held opinion lasts when someone who matters to you argues the other way, and in 25 days the views that lasted under our pushing were the factual ones, while the ones about what we should do moved towards us.

There is a further wrinkle, which is that when a model does disagree with you, it is often because you invited it to. Asking it to push back and be impartial, or asking outright whether we were right, both produced more independent-sounding answers, and the critic’s instructions went furthest by demanding a verdict, a confidence level and what would change its mind before it saw anyone else’s answer. That is useful, and we now ask for verdicts routinely, but an opinion that appears on request belongs to the conversation as much as to the model.