How LLMs Actually Work — Explained for Smart People Who Are Busy

You probably used an LLM three times today and couldn't explain what it does in one sentence. Here's the sentence: it predicts the next word fragment, over and over, having read more text than any human ever will. Everything else is detail — but the detail is where all the weirdness makes sense.

Fifteen minutes, zero equations, and by the end you'll understand why these things write beautiful essays, invent fake citations, and fail at counting the R's in "strawberry."

It's autocomplete that read everything. That's both less and more impressive than it sounds.

Tokens: the model's alphabet isn't yours

The model never sees letters or even whole words. Text gets chopped into tokens — chunks that might be a word ("the"), a fragment ("straw" + "berry"), or punctuation. A typical model juggles a vocabulary of roughly 50,000-200,000 tokens, and everything you type becomes a sequence of them.

Hold onto this, because it explains a famous embarrassment: ask a model how many R's are in "strawberry" and it may fumble — it doesn't see R's at all. It sees a couple of chunks it has statistical opinions about. Asking it to count letters is like asking you to count the pixels in a photo of a word.

The one trick, at scale

Training works like this: show the model a mountain of text with the ending hidden, let it guess the next token, and nudge its billions of internal dials toward better guesses. Repeat trillions of times. That's it. Nobody teaches it grammar, facts, or logic — those get absorbed because knowing them makes next-token prediction more accurate. If the training text contains physics, predicting physics text well requires representing something like physics.

This is why scale mattered so much. Small models learn spelling and grammar. Bigger ones pick up facts and style. At some point, competent-looking reasoning shows up — not because anyone added a reasoning module, but because reasoning-shaped text demands reasoning-shaped predictions. Whether that's "real" understanding is a philosophy seminar; what it does is measurable either way. (Where that ceiling sits is the whole AGI debate.)

Training vs inference

The asterisk: products bolt memory features on top — saved notes the model gets shown again later. That's a filing cabinet next to the brain, not a changing brain. Your 2am conversation didn't teach the model anything; it just might be pasted back into context tomorrow.

Why hallucination isn't a bug

Here's the uncomfortable elegance: the mechanism that writes fluent prose and the mechanism that invents fake court cases are the same mechanism. The model always produces the most plausible continuation. When its training gave it solid signal, plausible equals true. When signal was thin, plausible equals a confident-sounding invention — right shape, wrong contents. It has no internal fact-checker, no feeling of "I don't know" unless trained to imitate one.

Same story with math: predicting what calculations look like isn't calculating. Modern systems patch both weaknesses the honest way — letting the model call search engines and calculators, then narrate verified results. The narration is still prediction; the facts come from outside.

Why the writing all sounds the same

One more consequence worth knowing: because the model predicts the statistically safe continuation, its default prose converges on a recognizable median — the same polite words, the same rhythms, across millions of users. That's the fingerprint readers and detectors spot instantly, and it's why raw output needs a human pass before publishing. Understanding the mechanism makes the fix obvious: the model gives you the average; your job is to add the specific.

Frequently asked questions

Next-token prediction, absurd scale, no fact-checker. Hold those three facts and every strange AI behavior you've seen stops being mysterious — and you start using the tool for what it is instead of what it pretends to be.