Ask a language model a question and the answer appears in seconds. Understanding the machinery underneath is simpler than it sounds, and it explains both the impressiveness and the limits of the technology.
Tokens, not words
Text is split into tokens — word fragments the model can work with. Each token carries a numerical representation, and from that point the model deals only in numbers, never in language itself.
Prediction, repeated
The model's single job is to predict the next token given everything before it. It was trained on enormous amounts of text, adjusting internal weights whenever its prediction was wrong. What looks like reasoning is that prediction loop running thousands of times, one token at a time.
Why it sometimes gets things wrong
Because the output is plausible-sounding prediction rather than a database lookup, a model can state something false with complete confidence. It has no built-in notion of truth — only of what typically follows.
Temperature and creativity
A setting called temperature controls how daring the choices are. Low temperature sticks to the most likely continuation and stays factual; higher temperature mixes in unlikely tokens and produces more original, but riskier, text.
Where this is heading
Models are increasingly paired with tools — search, calculators, code execution — which converts their weakest trait, accuracy, into a strength by letting them verify before answering.