How does ChatGPT work?
A visual explanation

ChatGPT is a large language model: a neural network that, given some text, predicts what text comes next. It does this in small units called tokens.

1. Split the text into tokens — your message becomes a sequence of tokens: words, parts of words and punctuation.
2. Turn tokens into numbers — each token becomes a long list of numbers (an embedding) that captures something of its meaning.
3. Run the transformer — layers of the network let every token "attend" to every other, building up a representation of what the text means.
4. Score every possible next token — the output is a probability for each token in the vocabulary.
5. Pick one and repeat — a token is sampled, added to the text, and the whole process runs again until the reply ends.

The transformer architecture is published

2017

ChatGPT launches

30 Nov 2022

Average English token

~¾ word

Weekly ChatGPT users OpenAI reported in October 2025

800 million

A token is the unit of text the model reads and writes. Common words are often a single token; longer or rarer words are split into pieces. OpenAI's guidance is that one token is roughly four characters of English text, or about three-quarters of a word, so 100 tokens is about 75 words⁵. Text in Hindi, Tamil or other Indian languages usually needs more tokens per word, because tokenisers were built mostly from English-heavy data.

Thinking in tokens explains small oddities, such as why models used to struggle to count the letters in a word: they never saw the letters, only the tokens.

The transformer is the neural network design behind ChatGPT. It was introduced in 2017 by researchers at Google in a paper titled "Attention Is All You Need"¹. Its key mechanism, attention, lets the model, for each word, look at every other word in the text and decide how much each matters.

Earlier language models read text step by step and tended to forget the start of long sentences. Attention lets the connection between "it" and "metro" be made directly, however many words apart they are, and it can be computed for all words at once, which made it practical to train much bigger models on much more text. GPT stands for Generative Pre-trained Transformer: a model that generates text, is pre-trained on a large body of text, and uses this architecture.

ChatGPT is trained in two broad stages.

Pre-training. The model is shown enormous amounts of text and trained to predict the next token, over and over, adjusting billions of internal numbers (parameters) each time it is wrong. OpenAI's 2020 GPT-3 paper showed that a model trained this way at large scale, 175 billion parameters, could perform many tasks just from instructions and examples in the prompt². Pre-training gives a model broad knowledge of language and facts, but it produces a text predictor, not an assistant: ask it a question and it might continue with more questions.

Fine-tuning with human feedback. To turn the predictor into an assistant, OpenAI fine-tuned it on example conversations written by people and then used reinforcement learning from human feedback (RLHF): people ranked several model answers, a separate "reward model" learned their preferences, and the language model was trained to produce answers that score well³. In OpenAI's 2022 InstructGPT study, people preferred answers from a 1.3 billion parameter model trained this way over the 175 billion parameter GPT-3³. ChatGPT, launched on 30 November 2022, applied the same approach to dialogue⁴.

ChatGPT sometimes states false things fluently because it generates plausible text, not checked facts. When the right answer is rare or absent in its training, the most plausible continuation can be wrong but confident: a fake citation, a non-existent law, a wrong date. These errors are called hallucinations.

In a 2025 paper, researchers at OpenAI argued that hallucinations persist partly because of how models are graded: most training and evaluation reward a correct answer and give nothing for "I don't know", so guessing scores better than admitting uncertainty, like a student guessing on a multiple-choice test⁶. Their proposed fix is to change evaluations to penalise confident errors more than abstentions. OpenAI said in 2026 that newer default models produce markedly fewer hallucinated claims on high-stakes prompts, but the problem has not disappeared⁸.

ChatGPT and Google Search answer questions in different ways. Search retrieves and ranks existing pages from an index; a language model generates new text from what it learned in training. The two are converging: Google adds AI-generated overviews to results, and ChatGPT can run web searches and cite what it finds. Our guide to how Google Search works covers the retrieval side.

Table: How a search engine and a language model answer differently

• Design for being wrong — show sources, let people check, and make corrections easy.
• Ground answers in your own data — retrieval from your documents reduces invention on product-specific questions.
• Tell users what it can't do — clear limits build more trust than confident failures.
• Watch the tokens — long prompts and Indian-language text cost more tokens, which affects speed and price.
• Test with real users — people's trust in AI answers rises and falls quickly; measure it.

We cover the practical side in our guides to adding AI to an app and comparing ChatGPT, Claude and Gemini.

Large language model (LLM) — a neural network trained on large amounts of text to predict the next token.

Token — a chunk of text, often part of a word, that the model processes.

Transformer — the neural network architecture that uses attention to relate all parts of a text to each other.

Parameters — the internal numbers adjusted during training; large models have billions.

RLHF — reinforcement learning from human feedback: training a model on human preferences between answers.

Hallucination — a fluent, confident statement that is false.

The explanation follows the primary research papers: the transformer (Vaswani et al., 2017), GPT-3 (Brown et al., 2020), InstructGPT and RLHF (Ouyang et al., 2022) and OpenAI's 2025 analysis of hallucinations, plus OpenAI's own documentation on tokens and ChatGPT's launch. Usage figures are OpenAI's as reported by TechCrunch. The diagrams are our own simplified illustrations; probabilities and attention weights in them are illustrative. Last reviewed October 2026.