The One-Sentence Answer
ChatGPT predicts the next token — a small chunk of text — over and over again, billions of parameters deep, trained on an enormous slice of human writing. That one loop, repeated hundreds of times per reply, produces everything from sonnets to working Python code.
The rest of this article unpacks what that actually means, so you walk away with a genuine mental model — not just buzzwords.
Step 1: Text Gets Chopped Into Tokens
Before ChatGPT processes a single word, your message is split into tokens — the raw units the model works with. A token is roughly 3–4 characters, so common short words ("the", "is") are usually one token, while longer or rarer words get split into pieces. The word "unbelievable" might become un + bel + iev + able — four tokens.
Each token maps to a number (an ID), because neural networks speak math, not English. You can play with a live tokenizer in Lesson 8: Transformers & LLMs to see exactly how your own sentences get sliced up.
| Text example | Approximate tokens |
|---|---|
| "ChatGPT" | 3 tokens: Chat · G · PT |
| "Hello, world!" | 4 tokens |
| A 750-word essay | ~1,000 tokens |
| One page of a novel | ~600–700 tokens |
Step 2: The Transformer — How Tokens "Talk" to Each Other
Once your text is tokenized, it passes through a Transformer — the architecture that powers ChatGPT, Gemini, Claude, and most modern AI language systems. The key idea inside a Transformer is called self-attention, and it's what makes these models surprisingly good at understanding meaning.
Self-attention lets every token in a sentence look at every other token and ask: "How relevant are you to understanding what I mean?" The model assigns weights — higher weight to tokens that matter, lower to ones that don't — and uses those weights to build a richer representation of meaning.
This is a massive leap over older methods. Before Transformers, models processed text left-to-right, one word at a time, often losing track of context from earlier in a long sentence. Self-attention processes all tokens in parallel and can connect words hundreds of positions apart. If you want the linguistic background, Lesson 7 on Language & NLP covers how earlier approaches worked.
Step 3: What "Large" Actually Means
The "L" in LLM stands for Large — and it's not false advertising. ChatGPT's underlying models have hundreds of billions of parameters: numerical values (think adjustable dials) spread across many layers of the Transformer. During training, every dial is tuned to minimise one thing — the error on predicting the next token, across an enormous corpus of text from the internet, books, code, and more.
Training happens in two main stages:
- Pre-training: The model reads vast amounts of text and learns to predict what comes next. At this stage it knows a lot about language and facts, but it's not yet a helpful assistant — it might just continue your prompt in an unpredictable direction.
- Fine-tuning & RLHF: Human trainers rate responses for helpfulness, honesty, and harmlessness. Those ratings feed a second training pass (Reinforcement Learning from Human Feedback) that nudges the model toward being a useful conversational assistant rather than an unconstrained text predictor.
The Context Window: How Much Can It "See"?
Every conversation you have with ChatGPT is fed in as a block of tokens. The context window is the maximum number of tokens the model can look at in one go — its working memory, so to speak. Older versions had windows of around 4,000 tokens (roughly 3,000 words). Newer models support 128,000 tokens or more.
When a conversation exceeds the context window, the model can no longer "see" the earliest messages. It hasn't forgotten in the human sense — those tokens were simply never in view. This is why very long chats can feel like the model loses track of what was said at the start.
Why ChatGPT Makes Things Up (Hallucination)
This is the most important limitation to understand. ChatGPT is a probability machine, not a search engine or fact database. At every step it predicts the most plausible-sounding next token — and "plausible-sounding" is not the same as "true".
When the model doesn't have reliable information about something, it doesn't know to stop and say "I don't know." Instead it continues generating tokens that fit the pattern of a confident, fluent answer — producing text that sounds authoritative but can be completely wrong. This is called hallucination.
- Fabricated citations, made-up statistics, and non-existent laws are classic examples.
- Hallucinations are more likely with obscure topics, recent events (past the training cutoff), or highly specific factual claims.
- The model can also be right — often impressively so. The problem is you can't always tell which is which without checking.
What ChatGPT Is Not
A few common misconceptions are worth clearing up directly:
- It is not conscious or sentient. It processes token sequences and predicts outputs. There is no understanding, belief, or experience in the philosophical sense — it's a very sophisticated pattern-matcher.
- It is not browsing the internet (unless explicitly given a web-search tool). By default, ChatGPT's knowledge is frozen at its training cutoff date.
- It is not retrieving stored facts from a database. Every answer is generated fresh from the model's parameters — which is why identical questions can get slightly different answers.
- It is not AGI. General intelligence implies flexible reasoning across novel domains, autonomous goal-setting, and genuine understanding. Current LLMs are powerful but narrow — excellent at language tasks, brittle outside them.
For a broader look at what AI can generate, see Lesson 9: Generative AI and our article What Is Generative AI?
Frequently Asked Questions
Does ChatGPT understand what it says?
Not in the way humans do. ChatGPT has no internal model of the world, no intentions, and no awareness of meaning. It has learned extraordinarily rich statistical relationships between tokens — which can produce outputs that look like understanding. Whether that counts as a form of understanding is a genuine philosophical debate, but practically speaking: don't assume it grasps context the way a person would.
Why does ChatGPT make things up?
Because it's predicting plausible text, not retrieving verified facts. When the training data doesn't strongly constrain the answer, the model fills the gap with whatever token sequence fits the pattern of a fluent, confident response. There's no built-in "I'm not sure" alarm — though fine-tuning has improved this. Always verify important factual claims independently.
What is a token?
A token is the smallest unit of text the model works with — roughly 3–4 characters or about three-quarters of a word on average. Your entire conversation is converted into a sequence of token IDs (numbers) before the model ever runs. Tokens are not words: punctuation, spaces, and word fragments all count as tokens too.
Is ChatGPT the same as AGI?
No. AGI (Artificial General Intelligence) refers to a system that can learn and perform any intellectual task a human can, with genuine flexibility and understanding. ChatGPT is a large language model — exceptional at text tasks it was trained for, but lacking the autonomous reasoning, goal-setting, and broad adaptability that AGI would require. Researchers disagree about how close (or far) we are from AGI, but today's LLMs, impressive as they are, are not it.