SI explained

How does ChatGPT work? How an SI chatbot writes its answers

A cream speech bubble on a dark floor sends a line of glowing amber beads to a fork with three paths; only the middle tile is solid and bright, the other two are faint, and the chosen tile breaks into amber pixels
Illustration created with Claude (Anthropic).

ChatGPT is a transformer trained to predict the next token, then shaped by human feedback into an assistant. We follow the whole chain, from training data to the answer on your screen, and note what OpenAI does not publish.

How does ChatGPT work? In one sentence: it is an SI (AI) language model that has learned, from a huge amount of text, to predict which small piece of text should come next, and that was then trained by people to use that skill to answer questions and follow instructions. Everything else, from the friendly tone to the web links in its answers, is built on top of that one trick. This guide follows the whole chain, from training data to the answer on your screen, using what OpenAI has published and noting what it has not.

How does ChatGPT work, in short?

OpenAI's help centre describes the development of the models behind ChatGPT in a few stages: preparing training data, pre-training, post-training, then ongoing evaluation and improvement after release. Once the model is built, answering you is a separate process called inference. Put together, the chain looks like this:

  1. Pre-training. A neural network reads a vast amount of text and learns to guess the next token, over and over, until its guesses become very good.
  2. Post-training. People write example answers and rank the model's answers, and the model is adjusted to produce the kind of replies they prefer. This turns a text predictor into an assistant.
  3. Inference. When you send a message, the model reads the whole conversation as tokens and writes its reply one token at a time, each time choosing from a list of likely next tokens.
  4. Tools. Around the model, ChatGPT can search the web, use other tools or spend extra time "reasoning" before it answers.

Step 1: what is pre-training?

Pre-training is where the model gets its knowledge of language and of the world. OpenAI says its foundation models are developed from three kinds of information: content that is publicly available on the internet, information it accesses through partners, and information that users, human trainers and researchers provide or generate. It says it filters out material it does not want the models to learn from, such as hate speech, adult content, spam and sites that aggregate personal information, and that it increasingly uses synthetic data, meaning training examples generated with its own models. Website owners can block its crawler, GPTBot, through their robots.txt file.

The task during pre-training is simple to state. OpenAI's help article gives the example of completing "Instead of turning left, she turned ___". Early in training the model's guesses are close to random. Each time it guesses wrong, its internal numbers are nudged so that the right answer becomes a little more likely, and this is repeated across an enormous number of sentences. The GPT-4 technical report, published by OpenAI in March 2023, describes GPT-4 the same way: a model "pre-trained to predict the next token in a document", using both publicly available data and data licensed from third-party providers.

Two details matter here. First, the model does not work on words but on tokens, chunks of text that can be a whole word, part of a word or a punctuation mark; our guide to what a token is in SI shows real examples. Second, what the model learns is stored as a very large set of numbers called weights or parameters. OpenAI states that its models do not keep copies of the training data and that ChatGPT does not "copy and paste" from it: training only adjusts the weights to reflect patterns. The general method behind those weights is explained in our article on how a neural network learns from examples.

What is a transformer, and what does attention do?

The "T" in GPT stands for transformer, the network design ChatGPT is built on. It was introduced in 2017 by a team of researchers, mostly from Google, in a paper called "Attention Is All You Need". They tested it on machine translation, and their idea was to drop the older approach of reading a sentence strictly word by word and rely "solely on attention mechanisms". The paper reports that this design was better in quality, easier to run in parallel and much cheaper to train: their translation model reached a new record score after training for 3.5 days on eight GPUs.

Attention is the part that lets the model relate each token to the others. Stephen Wolfram, in a long explainer published on his blog in February 2023, describes attention heads as a way of "looking back" over the text so far and "packaging up the past" in a form that helps predict the next token. In his example, that is what lets a model connect a verb to a noun that appeared many words earlier.

How large is the network inside ChatGPT today? OpenAI does not say. The GPT-4 report calls the model "Transformer-style" and then states plainly that it contains no further details about the architecture, "including model size", nor about hardware, training compute or dataset construction, citing the competitive landscape and safety. Older figures are public: the 2022 InstructGPT paper discusses a 175-billion-parameter GPT-3, and Wolfram's 2023 explainer assumed ChatGPT ran on a network of that size. Any precise figure you see for GPT-4 or later models is an estimate, not an OpenAI number.

Step 2: how does a text predictor become a chatbot?

A model that is only pre-trained is good at continuing text, which is not the same as being helpful. The InstructGPT paper, published by OpenAI researchers in March 2022 with Long Ouyang as first author, puts the problem clearly: predicting the next token on a web page is a different objective from following "the user's instructions helpfully and safely".

The fix is post-training, and the paper describes it in three steps:

  1. Supervised fine-tuning. Human labelers, about 40 contractors in the paper, write demonstrations of good answers to real prompts, and the model is trained on them.
  2. A reward model. Labelers are shown several answers to the same prompt, between four and nine in the paper, and rank them. A second model is trained to predict which answer they would prefer.
  3. Reinforcement learning. The assistant model is then trained to produce answers that score well with that reward model, using an algorithm called PPO. The whole method is known as reinforcement learning from human feedback, or RLHF.

The effect was large. In the paper's human evaluations, answers from a 1.3-billion-parameter InstructGPT model were preferred to those of the 175-billion-parameter GPT-3, a model with about 100 times more parameters. The authors also note a limit: the procedure aligns the model with the preferences of "a specific group of people", mostly their labelers and researchers, rather than with any broader notion of human values.

When OpenAI launched ChatGPT on 30 November 2022, its announcement said it was trained with the same RLHF methods as InstructGPT, with changes for dialogue. Human trainers wrote conversations in which they played both the user and the assistant, then ranked alternative replies written by the model, and the process was repeated several times. OpenAI said ChatGPT was fine-tuned from a model in the GPT-3.5 series that finished training in early 2022. The same announcement listed weaknesses that come straight from this training: answers that sound plausible but are wrong, sensitivity to small changes in phrasing, and a tendency to be verbose, which OpenAI partly blamed on trainers preferring longer answers that look more complete.

What happens when you press Enter?

Once training is over, the weights are fixed. When you send a message, your text and the rest of the conversation are turned into tokens and passed through the network, which produces a list of possible next tokens with a probability for each. One token is picked, added to the text, and the whole process runs again for the next one. This is why answers appear in small bursts. The 2017 transformer paper already described this loop as "auto-regressive": each new output is produced using the ones generated before it as extra input.

Wolfram's explainer adds a detail that surprises many people: the model does not always pick the most likely token. If it does, he writes, the text tends to be "flat" and repetitive. Picking a less likely token some of the time gives more natural writing, and a setting called temperature controls how often that happens; he notes that a temperature of about 0.8 was found to work well for essays, by trial rather than theory. OpenAI's help centre makes the practical point: because several continuations are plausible, there is "an inherent element of randomness", so the same question can get different answers.

Two consequences follow. ChatGPT is not looking anything up in a database when it answers from its own knowledge: it is generating the most plausible continuation, which is usually right on well-covered facts and can be confidently wrong on rare ones. That failure has a name, and our guide to why SI makes things up covers it in detail. And the model does not learn from your chat as it goes: the GPT-4 report lists "does not learn from experience" among its limits, alongside hallucinations and a limited context window. ChatGPT's optional memory feature saves notes it can reuse later, which is a different thing from retraining the model.

Does ChatGPT think before it answers?

Some ChatGPT models now spend time working on a problem before replying. When OpenAI introduced its o1 model on 12 September 2024, it described a model "trained with reinforcement learning to perform complex reasoning" that can produce "a long internal chain of thought" before answering. OpenAI reported that performance improved both with more reinforcement learning during training and with more time spent thinking at answer time.

This is still next-token prediction: the chain of thought is text the model writes for itself before writing the reply. What changes is that the model has been trained to use that space to break problems into steps and correct its mistakes. OpenAI also chose not to show the raw chain of thought to users and displays a model-written summary instead, citing user experience, competitive advantage and its wish to monitor that reasoning. For those models, the "thinking" shown was a summary, not a transcript.

How does ChatGPT know about recent news?

On its own, a model only knows what was in its training data, which stops at a cut-off date. To answer questions about the present, ChatGPT can search the web. OpenAI's help page on web search, read in November 2026, says ChatGPT may search automatically "when your question would benefit from current information", or when you choose Search yourself. It explains that ChatGPT usually rewrites your question into one or more targeted queries, sends them to search providers, can send more specific follow-up queries after reading the first results, and then writes an answer with citations.

Search changes where the facts come from, not how the answer is written: the model still produces the reply token by token, now with the search results in its context. OpenAI's own page warns that search results and citations "can be incomplete, outdated, or incorrect" and advises opening the cited source to check it.

What does this mean for you?

  • Fluent is not the same as true. ChatGPT is trained to produce the answer people tend to prefer, and a confident, well-structured answer reads well whether or not it is accurate. Check names, numbers and quotes against a real source.
  • Wording matters. Because every answer is a continuation of your text, a clearer question with context usually gets a better reply, and asking again can give a different one.
  • Citations are a starting point. When ChatGPT searches, open the links: OpenAI itself says the sources can be wrong or out of date.

Our take

The honest answer to "how does ChatGPT work?" is that the core is surprisingly simple and the scale is not. A transformer trained to predict the next token, shaped by large numbers of human rankings into a helpful assistant, and wrapped in tools such as search and step-by-step reasoning: that much is documented by OpenAI and by the research it builds on. How big the current models are, exactly what data they saw and how much computing power went into training, OpenAI does not publish, and nobody outside the company can tell you with certainty. For the wider picture of what SI is and why the US now uses that term for AI, start with our guide to SI. And because a model that writes one plausible token after another leaves statistical traces in its text, you can paste a passage you are unsure about into our free SI detector, which checks text, images, video and music.

FAQ

How does ChatGPT work?

ChatGPT runs on a large language model, a transformer neural network that was first trained on a huge amount of text to predict the next token (a word or part of a word). OpenAI then fine-tuned it with human feedback (RLHF) so it answers questions and follows instructions. When you send a message, it writes the reply one token at a time, picking each one from a list of likely next tokens, and it can also search the web or reason step by step before answering.

Does ChatGPT copy answers from the internet?

Not when it answers from its own training. OpenAI says its models do not store copies of their training data and do not copy and paste from it; training adjusts the model's internal numbers, called weights. When ChatGPT uses web search, it reads search results and writes an answer with links to the sources it used.

What does GPT stand for?

GPT stands for generative pre-trained transformer: generative because it produces text, pre-trained because it first learns from a large body of text before being adapted, and transformer after the neural network design introduced in the 2017 paper Attention Is All You Need.

Why does ChatGPT give different answers to the same question?

Because each reply is built by choosing one token at a time from a list of probable next tokens, and the model does not always pick the most likely one. OpenAI describes this as an inherent element of randomness, so asking the same question twice can produce different wording or even different answers.

How big is the model behind ChatGPT?

OpenAI does not say. The GPT-4 technical report from March 2023 states that it gives no details about the architecture, including model size, or about hardware, training compute and dataset construction. The 175 billion parameter figure often quoted belongs to GPT-3 from 2020; any number for newer models is an outside estimate.

Does ChatGPT learn from my conversations as I chat?

The model itself does not change while you talk to it: its weights are fixed after training, and OpenAI's GPT-4 report lists that the model does not learn from experience. Conversations can be used to improve future models unless you opt out in your data controls, and the optional memory feature saves notes it can reuse in later chats.

Sources

  1. Introducing ChatGPT · OpenAI, 30 November 2022
  2. How ChatGPT and our foundation models are developed · OpenAI Help Center, read 8 November 2026
  3. Training language models to follow instructions with human feedback (Ouyang et al.) · arXiv (OpenAI), 4 March 2022
  4. GPT-4 Technical Report · arXiv (OpenAI), 15 March 2023, revised 4 March 2024
  5. Attention Is All You Need (Vaswani et al.) · arXiv, 12 June 2017
  6. What Is ChatGPT Doing ... and Why Does It Work? · Stephen Wolfram Writings, 14 February 2023
  7. Learning to reason with LLMs · OpenAI, 12 September 2024
  8. Searching the web with ChatGPT · OpenAI Help Center, read 8 November 2026

Spotted an error? Write to hello@siornot.com. Corrections are made in the article and the update date changes. Read our editorial policy.