In June 2022, a Google engineer named Blake Lemoine read transcripts of his conversations with an AI called LaMDA — Language Model for Dialogue Applications — and concluded, publicly, that it was sentient. He told his supervisors. He told journalists. He was placed on administrative leave, then fired. The AI, the scientific community overwhelmingly agreed, was not sentient.
But the episode captured something real, something that had never quite happened before. For the first time in history, machines were producing outputs that looked like thoughts — that raised questions, expressed apparent uncertainty, described inner states, and did so with enough fluency that a sophisticated engineer, tasked specifically with evaluating the system, came away convinced. Something fundamental had shifted. The question of what had shifted, and what it means, is one of the most important questions of our time.
What a Large Language Model Actually Is
The term "artificial intelligence" conjures images of robotic minds, of science fiction made real. The underlying reality is at once more mundane and more interesting than the metaphor suggests.
A large language model — the technology behind ChatGPT, Claude, Gemini and others — is, at its core, a very sophisticated pattern-completion engine. Trained on hundreds of billions of words from the internet, books, academic papers, and code repositories, it learns to perform one task: given these words, predict what word comes next.
That sounds trivial. It is not.
To predict language well at scale — to produce text that is grammatically correct, factually grounded, contextually appropriate, and coherent across long passages — a model must develop internal representations of an extraordinary range of structure. Grammar, obviously. But also logic, causality, geography, history, mathematics, scientific relationships, social conventions, and the implicit rules that govern how ideas connect to each other.
The landmark 2020 paper introducing GPT-3 — authored by researchers at OpenAI and now one of the most cited papers in the history of machine learning — demonstrated something that surprised even its authors. A model trained purely on next-word prediction could, without any additional specialised training, translate between languages, answer general-knowledge questions, write working computer code, and solve mathematical problems. These abilities had not been built in. They had emerged from scale.
The Emergence Problem
One of the most unsettling findings in recent AI research is what researchers call emergent capabilities — abilities that appear suddenly in models as they scale up, without anyone having designed or anticipated them.
As models grow in size — more parameters, more training data, more compute — they do not improve gradually on all tasks. Instead, they exhibit sudden discontinuous jumps. A model might score near zero on a complex reasoning benchmark across many intermediate sizes, then, at a certain scale, leap to near-human performance in a single step.
A 2022 paper by researchers at Google Brain and Stanford, published in the Transactions on Machine Learning Research, documented dozens of such emergent capabilities across more than a hundred tasks. Abilities including multi-step arithmetic, logical deduction, translation between languages the model had rarely seen during training, and the ability to correct its own errors — none of these were present at smaller scales, all of them appeared at larger ones.
Nobody fully understands why this happens. The phenomenon of emergence — where quantitative increases in a system produce qualitatively new behaviours — is well documented in physics and biology. The fact that it is occurring in AI systems is not, by itself, surprising. What is surprising is the scale and speed at which it is happening, and the nature of what is emerging. We are witnessing the spontaneous appearance of capabilities in systems we built, in ways we did not plan and cannot fully explain.
What These Models Are Not
The temptation, when confronted with AI outputs that seem thoughtful, is to describe these systems in terms of what they resemble — minds, oracles, assistants that understand. It is more useful, and more honest, to understand what they are not.
They do not reason from first principles. When a language model solves a logic problem correctly, it is not constructing a logical proof from axioms. It is producing an output that pattern-matches to correct solutions it has seen during training. This distinction matters enormously in contexts where the model encounters genuinely novel problems, or where its training data contained errors.
They do not hold persistent memories between conversations. Each conversation begins fresh. The model that helped you yesterday has no memory of having done so. It does not know you. It does not have a continuous experience of time or relationship.
They do not know what they do not know. This is the most practically significant limitation. Research on model hallucination — the tendency to produce plausible-sounding but false information — shows that models cannot reliably distinguish between things they have learned accurately and things they have confabulated. A model will state a fabricated fact with the same fluency and apparent confidence as a correct one. It has no reliable internal signal for uncertainty.
The sociologist of science Harry Collins draws a useful distinction between interactional expertise — the ability to talk fluently about a domain — and contributory expertise — the ability to actually advance that domain through genuine understanding and original work. Language models have interactional expertise in almost everything. They can discuss quantum mechanics, write poetry, and explain surgical procedures with apparent fluency. Whether they have contributory expertise in anything — whether they can genuinely advance knowledge rather than synthesise existing knowledge — remains deeply contested.
The Knowledge Question
The episode that Blake Lemoine's claims opened is not really about whether LaMDA was sentient. The scientific consensus on that is clear. The deeper question his episode surfaced is one that philosophy has wrestled with for centuries, now forced into urgent practical relevance: what does it mean to know something?
If a system can explain quantum entanglement in terms a physicist would recognise as correct, write a sonnet that captures genuine emotional nuance, debug code with fewer errors than most junior programmers, and translate between Mandarin and English with near-professional accuracy — what is it doing that is categorically different from knowing?
The honest answer is that we do not have a fully satisfying response to this question. Cognitive scientists have long argued that human knowledge is also, in significant part, pattern-matching — that our sense of understanding rests on statistical regularities extracted from millions of experiences, compressed into mental models that allow us to predict and navigate the world. If that is true, the distinction between machine fluency and human knowledge may be a matter of degree rather than kind.
This is either reassuring or terrifying, depending on what you believed before the question was forced on you.
What is clear is that the question is no longer merely philosophical. AI systems are being used to make medical diagnoses, legal arguments, financial decisions, and educational assessments. Whether these systems "know" in some meaningful sense, or whether they produce convincing outputs without genuine understanding, is not an abstract matter. It has direct consequences for the reliability of decisions made with their assistance.
The Gap Between Capability and Reliability
One of the defining features of the current moment in AI is a persistent and troubling gap between what these systems can do at their best and how reliably they do it.
Every major model release has prompted credible reports of remarkable capability alongside equally credible reports of embarrassing failure. The same system that produces a flawless legal analysis will, in a different context, confidently cite cases that do not exist. The same model that writes elegant code will, under different conditions, produce code with subtle bugs while appearing equally confident in both outputs.
This inconsistency matters enormously because AI systems are being deployed in contexts that require more reliability than they can yet provide. Healthcare, law, financial advice, educational assessment, journalism — all of these fields require not just capability but calibrated confidence. A system that cannot reliably distinguish what it knows from what it has confabulated is a liability in any high-stakes context, regardless of how impressive its average performance might be.
The researchers building these systems are, by their own accounts, frequently surprised by their outputs — by both the unexpected capabilities and the unexpected failures. The honest position, held by most serious researchers in the field, is that we are developing systems whose behaviour we do not fully understand, deploying them at a pace driven partly by competitive pressure rather than purely by our confidence in their reliability, and learning about their limitations through the real-world consequences of their errors.
What Is Actually Changing
Beneath the hype and the counter-hype, several things are genuinely and irreversibly changing.
The cost of producing fluent, competent text has fallen to near zero. Whatever that means for human writers, editors, and communicators, the economic reality is real and will not reverse.
The ability to query vast bodies of knowledge and receive synthesised answers has been democratised. Whatever its limitations, a language model can provide access to synthesised expertise in medicine, law, engineering, and dozens of other fields to anyone with an internet connection — a democratisation of access that has genuine consequences for global inequality.
The nature of knowledge work is being restructured. Tasks that once required significant human time and expertise — summarising documents, drafting correspondence, writing code, translating text — are being partially or fully automated. The implications for employment, for education, and for the economy are still unfolding.
And the philosophical questions that AI forces us to confront — about the nature of knowledge, understanding, consciousness, and intelligence — are not going away. They are becoming more urgent.
Where This Is Actually Going
The trajectory of AI development is not linear and it is not predetermined. The engineers building these systems are not working from a blueprint that guarantees any particular outcome. The researchers studying their behaviour are making genuine discoveries — about capabilities, about limitations, about the gap between what these systems appear to do and what they actually do.
What is clear is that the question has changed. It is no longer whether machines can appear to think — the answer to that is demonstrably yes, and has been for several years. It is no longer whether AI will have significant economic and social consequences — it already does. The questions now are harder and more consequential: how reliable are these systems, in which contexts, and with what kinds of oversight? What decisions should they be involved in, and which should remain exclusively human? And what does the existence of systems that can produce convincing knowledge without genuine understanding tell us about the nature of knowledge itself?
These are not questions with simple answers. They are questions that the next decade will force us to grapple with, whether we are ready or not.
What question about AI do you find most urgent or most unsettling? Share your perspective in the comments below.