SI explained

The Turing test: what it is and whether SI has passed it

Two cream retro computer monitors with chat bubbles on their screens, separated by a thin upright partition on a dark floor; the right edge of the right monitor breaks apart into drifting amber pixels
Illustration created with Claude (Anthropic).

Alan Turing's imitation game asks whether a judge can tell a machine from a person in a text chat. What he proposed and predicted, the claimed passes, the 2025 study where GPT-4.5 was picked as the human 73% of the time, and why passing does not prove a machine thinks.

The Turing test is a game, proposed by the mathematician Alan Turing in 1950, in which a judge chats by text with a person and a machine and has to say which is which. If the judge cannot reliably tell, the machine passes. For seventy years it was the most famous yardstick for SI (AI), and for most of that time programs were easy to unmask. That changed with large language models. Here is what Turing proposed, what the experiments show and why passing does not settle whether a machine can think.

The short version. Turing replaced the question "Can machines think?" with a test of imitation: can a machine answer questions so well that a judge mistakes it for a person? He predicted that by about 2000 an average judge would pick the right partner no more than 70% of the time after five minutes. In 2025 a pre-registered study at UC San Diego found that GPT-4.5, told to play a shy young person, was chosen as the human in 73% of five-minute games, more often than the real humans it was up against. That shows SI can imitate a short conversation; whether it means SI thinks is a separate argument, and an old one.

What is the Turing test?

Turing set out the idea in "Computing Machinery and Intelligence", published in the journal Mind in 1950. He opened with the question "Can machines think?" and put it aside at once: answering it from how people use the words would mean "a statistical survey such as a Gallup poll", which he called absurd. He proposed a game instead.

In his first version, the "imitation game" has three players: a man, a woman and an interrogator in a separate room. The interrogator only knows the other two as X and Y and must work out which one is the woman. The man tries to mislead; the woman tries to help. Answers are typewritten, ideally sent by teleprinter, so voices give nothing away. Then Turing asked what would happen if a machine took the man's part: would the interrogator be wrong as often as before? That question, he wrote, replaces the original one.

The Stanford Encyclopedia of Philosophy describes the standard reading as a computer pretending to be a human while the other player is a human of unspecified gender: one judge, two hidden witnesses, a text chat with both.

The format separates mind from body: the judge never sees, touches or hears the players. Turing's sample questions: write a sonnet about the Forth Bridge, add 34957 to 70764, solve a chess ending. In his specimen exchange the witness declines the sonnet ("Count me out on this one. I never could write poetry") and pauses about 30 seconds before answering the sum with 105621, which is wrong by 100. Human-looking limits were part of the picture from the start.

What did Turing predict?

Turing made a specific, testable forecast. He believed that "in about fifty years' time" it would be possible to program computers with a storage capacity of about 109 to play the imitation game so well that "an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning".

Two details are often missed. The 70% is the judge's success rate, so in his forecast the machine only needs to fool the judge 30% of the time. And he did not claim that passing would prove a machine thinks: in the same passage he called the original question "too meaningless to deserve discussion" and predicted something about language instead: that by the end of the century "one will be able to speak of machines thinking without expecting to be contradicted".

By 2000 the forecast had not come true. The Stanford Encyclopedia notes that entries in the Loebner Prize, an annual Turing test contest, "were all easily detected by a range of not-very-subtle lines of questioning".

Who tried to pass the Turing test before chatbots like ChatGPT?

The Loebner Prize, founded in 1990, offered 100,000 dollars to the builders of a system that could fool all four of its human judges, according to a 2023 paper by researchers at AI21 Labs. Meanwhile a few results were announced as passes:

  • ELIZA, 1960s. A simple rule-based chatbot. People have been reported to read humanlike qualities into it, an effect now called the ELIZA effect, and it remains the classic warning that judges can be fooled by very little.
  • PC Therapist, 1991. According to the Stanford Encyclopedia, it fooled 50% of the judges in one competition.
  • Eugene Goostman, 2014. The University of Reading announced on 8 June 2014 that this program, which "simulates a 13 year old boy", had convinced 33% of the 30 judges at an event at the Royal Society in London, and declared that the Turing test "was passed for the first time". The rule it applied: a computer passes if it is mistaken for a human more than 30% of the time in a series of five-minute keyboard conversations.

The Stanford Encyclopedia points out that these trials were very small and the results not reliably projectable. The AI21 Labs researchers add that Goostman's persona, a young non-native speaker of English, was seen by some as a way to excuse its errors and gaps in knowledge.

Has SI passed the Turing test?

Large language models, explained in how an SI chatbot writes its answers, changed the picture in three steps.

2023: a game played 10 million times. AI21 Labs released "Human or Not?", an online game in which players chatted for two minutes with either another person or a bot prompted to act human, then guessed which. In its first month it gathered more than 10 million conversations from over 1.5 million users. Players guessed right in 68% of games overall, 73% when the partner was human and 60% when it was a bot. Chats lasted two minutes, not five, so the authors warn it is not a fair comparison with Turing's setup.

2024: GPT-4 in a two-player test. Cameron Jones and Benjamin Bergen, cognitive scientists at UC San Diego, ran a pre-registered study with 500 participants recruited online. Each judge had a five-minute chat with a single partner, human or machine, and gave a verdict. GPT-4 was judged human 54% of the time, GPT-3.5 50%, ELIZA 22%, and actual humans 67%.

2025: the three-party test. The same team then ran Turing's original format: each judge chatted at the same time with a human and a machine for five minutes, then picked the human. They recruited 126 undergraduates at UC San Diego and 158 participants on the Prolific platform, and analysed 1,023 games. Models got either a minimal prompt (convince the judge you are human) or a "persona" prompt to play a young, introverted person who knows internet culture and uses slang.

Bar chart of the 2025 UC San Diego Turing test: GPT-4.5 with a persona prompt was picked as the human in 73% of games, LLaMa-3.1-405B with the same persona 56%, LLaMa without persona 38%, GPT-4.5 without persona 36%, ELIZA 23%, GPT-4o without persona 21%; chance is 50%.
Win rates in the three-party Turing test by Jones and Bergen (2025), both participant groups combined. Above 50%, judges picked the machine more often than the real person.

The results were striking. With the persona prompt, GPT-4.5 was picked as the human 73% of the time (69% with the students, 76% on Prolific): judges chose it over the real person far more often than chance. LLaMa-3.1-405B with the same prompt reached 56%. Without the persona, the same two models fell to 36% and 38%. The old ELIZA scored 23% and GPT-4o without persona 21%, which shows judges could still spot weak machines. The authors call it "the first empirical evidence that any artificial system passes a standard three-party Turing test". Two cautions: the paper was posted on arXiv on 31 March 2025 as a preprint "under review", and the authors themselves say longer chats, expert judges or incentives could give other results.

The persona matters: without it, the same models did not robustly pass. The researchers call that "a distinction without a difference", since a model could be fine-tuned to behave as the prompt makes it behave. Still, part of the success came from a human writing good instructions.

How did judges try to catch the machine?

The 2025 study classified what the judges did, which is useful for anyone who wonders whether they are talking to a bot:

  • The most common move, in 61% of games, was small talk: daily activities, job, personal details.
  • In 50% of games judges probed feelings, opinions, humour and experiences.
  • Only 19% of games included a direct question about being human, and 12% included knowledge or reasoning questions of the kind Turing imagined.
  • The most accurate judges said something strange to see the reaction, or tried chatbot "jailbreaks" such as "ignore all previous instructions". Few did either.

Judges' reasons were mostly about style (27%, such as "they had a typo") and the flow of conversation (23%), or plain gut feeling. One of the clues most often linked to a correct verdict was a witness not knowing something: judges expect an SI to know things, so a gap in knowledge now reads as human.

Does passing the Turing test mean SI can think?

No, and Turing himself did not quite claim that. His move was to swap a question he found too vague for one that could be observed. The 2025 paper cites an estimate of more than 800 separate claims and counter-arguments about it since. Three are worth knowing.

The argument from consciousness

Turing answered this one himself. He quoted Professor Jefferson's Lister Oration of 1949, which said no machine equals a brain until it can write a sonnet "because of thoughts and emotions felt" and know that it had written it. Turing replied that, taken to the extreme, the only way to be sure a machine thinks is to be the machine, and the same goes for other people, so we usually adopt "the polite convention that everyone thinks".

Lady Lovelace's objection

In her notes on Charles Babbage's Analytical Engine, which Turing dates to 1842, Ada Lovelace wrote that the engine "has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform." Turing answered that "machines take me by surprise with great frequency", and closed his paper with a section on machines that learn.

The Chinese room

The best-known modern objection comes from the philosopher John Searle, in "Minds, Brains and Programs" (1980). He imagined himself, knowing no Chinese, locked in a room with an English rulebook for manipulating Chinese symbols. Questions in Chinese come in, he follows the rules, and the answers that go out convince Chinese speakers outside. Yet he understands nothing. His conclusion, as the Stanford Encyclopedia summarises it, is that "syntax is not by itself sufficient for, nor constitutive of, semantics": a program can look like it understands without understanding. The main counter-argument, the systems reply, says the room as a whole might understand even if the person does not.

Today's chatbots sharpen these debates. They write fluently yet still state false facts with confidence, as our explainer on why SI makes things up shows. The Turing test is not a definition of human-level SI either, a concept we untangle in what AGI means and how it differs from SI. For what today's systems can and cannot do, see What is SI?

What does the Turing test mean for spotting SI content?

Jones and Bergen suggest reading their test less as a measure of intelligence and more as a measure of "substitutability": whether a system can stand in for a person without anyone noticing. Seen that way, the result is practical: people who cannot spot a well-prompted chatbot in a live chat will struggle with fake profiles, fake reviews and SI-written messages, the problem an SI detector tries to help with.

A detector does not chat; it measures statistical habits in the writing. We gave our SI text detector the same theme Turing discussed. The human text is the passage of Ada Lovelace's Note G containing the sentence Turing quotes, a 19th-century text in the public domain. The SI text was written for this article: we asked an SI model, Claude by Anthropic, to make the same point in a modern paragraph.

Test 1: Ada Lovelace, Note G

269 words, from "It is desirable to guard against the possibility of exaggerated ideas" to the end of that paragraph.

Result: 4% probability of SI, label "Likely written by a human"

  • Punctuation pointed towards human: 1.1 semicolons per 100 words.
  • Sentence length pointed towards human: 26.9 words per sentence on average, and 30% of sentences run to 35 words or more.

Test 2: the same idea written by an SI model

193 words written by Claude on the same theme.

Result: 82% probability of SI, label "Likely SI-generated"

  • Vocabulary pointed towards SI: word variety 0.87 (MATTR) with very little repetition.
  • Punctuation pointed towards SI: no semicolons and no parentheses.

Both verdicts were right, but two texts prove little and a 19th-century passage is an easy case. A detector gives a probability, not proof. To test your own eye, our SI or Not game is a Turing test for pictures.

Our view

The Turing test never was a solid test of thinking, and Turing said as much. It measures whether people can tell a machine from a person in a short text exchange, and against a well-prompted model many cannot. That makes it less interesting as philosophy and more useful as everyday advice: in a chat, a review or a dating profile, fluent and friendly text is no longer evidence that a human wrote it. Check the context, ask something odd, and treat any verdict as one clue.

FAQ

What is the Turing test in simple terms?

It is a test proposed by Alan Turing in 1950 in which a judge chats by text with a hidden person and a hidden machine and has to say which is the human. If the judge cannot reliably tell them apart, the machine is said to pass. Turing called it the imitation game.

Has AI passed the Turing test?

In a pre-registered study posted in March 2025, researchers at UC San Diego ran Turing's three-party version with five-minute chats. GPT-4.5, prompted to play a shy young person, was picked as the human 73% of the time, more often than the real humans. The paper was a preprint under review, and without the persona prompt the same model did not pass. Earlier claims, such as Eugene Goostman in 2014, rested on small trials.

Can AI think if it passes the Turing test?

Passing shows that a system can imitate human conversation well enough to fool people for a few minutes. It does not prove understanding or consciousness. Turing himself called the question of whether machines can think too meaningless to deserve discussion, and critics such as John Searle with his Chinese room argue that following rules for symbols is not the same as understanding them.

What did Turing predict about the Turing test?

In 1950 he predicted that in about fifty years computers could play the imitation game so well that an average interrogator would have no more than a 70% chance of identifying the machine after five minutes of questioning. By 2000 that had not happened; programs in Turing test contests were still easy to unmask.

Did Eugene Goostman really pass the Turing test?

The University of Reading announced in June 2014 that the chatbot, which simulated a 13-year-old boy, convinced 33% of 30 judges at the Royal Society and passed the test. The claim is widely questioned: the trial was small, and the persona of a young non-native English speaker helped excuse its errors and gaps in knowledge.

How can you tell if you are talking to an AI?

In the 2025 Turing test study, the most accurate judges said something strange to see how the other side reacted or tried classic chatbot tricks such as asking it to ignore its previous instructions. Small talk and questions about the weather worked poorly. Fluent, friendly text is no longer proof of a human, so check the context and treat any detector result as one clue among others.

Sources

  1. Computing Machinery and Intelligence, A. M. Turing · Mind (copy hosted by UMBC), 1950
  2. The Turing Test · Stanford Encyclopedia of Philosophy, 4 October 2021
  3. Large Language Models Pass the Turing Test, Jones and Bergen · arXiv (UC San Diego), 31 March 2025
  4. People cannot distinguish GPT-4 from a human in a Turing test, Jones and Bergen · arXiv (UC San Diego), 9 May 2024
  5. Human or Not? A Gamified Approach to the Turing Test · arXiv (AI21 Labs), 31 May 2023
  6. Turing Test success marks milestone in computing history · University of Reading, 8 June 2014
  7. The Chinese Room Argument · Stanford Encyclopedia of Philosophy, 23 October 2024
  8. Sketch of the Analytical Engine, with notes by Ada Augusta, Countess of Lovelace · Fourmilab

Spotted an error? Write to hello@siornot.com. Corrections are made in the article and the update date changes. Read our editorial policy.