The Core Idea: Predicting the Next Piece
At its core, a large language model (LLM) generates text by repeatedly predicting the most likely next token (roughly a word or word-piece) given everything that came before it. It does this one token at a time, feeding each new token back in as context for predicting the next one, until it produces a complete response.
How It Learns to Predict Well
During training, a model is shown enormous amounts of text and learns to adjust billions of internal numerical values (parameters) so its predictions get progressively closer to real, coherent text. This process teaches it grammar, facts, reasoning patterns, and style, purely from exposure to patterns in the training data — not from explicit rules a person wrote.
The Transformer Architecture
Most modern LLMs are built on an architecture called the transformer, which uses a mechanism called "attention" to weigh how relevant every other word in the context is to predicting the next one. This lets the model handle long-range relationships in text — connecting a pronoun back to a noun mentioned several sentences earlier, for instance — far more effectively than older architectures.
Why This Explains Both Its Strengths and Its Limits
Because an LLM is fundamentally predicting plausible next tokens based on learned patterns, it can produce remarkably fluent, contextually appropriate text — but it can also produce confident, fluent-sounding text that's factually wrong (see our Why Do AI Models Hallucinate? page), since fluency and factual accuracy aren't the same thing to the underlying prediction process.
Related Pages
Frequently Asked
Does an LLM think the way a person does?
Not in the way most people mean by 'think' — it's predicting likely next tokens based on learned statistical patterns, which can produce very human-like output without necessarily reflecting human-like reasoning underneath.
What does 'large' in large language model refer to?
It generally refers to the scale of the model (its number of parameters) and the size of the dataset it was trained on, both of which are typically enormous for modern models.
Why do LLMs sometimes make things up?
Because they're generating the most statistically plausible continuation of text, not retrieving verified facts from a database; see our Hallucination page for more detail.
What's the transformer architecture?
It's the neural network design underlying most modern LLMs, built around a mechanism called attention that helps the model weigh which earlier words matter most for predicting the next one.