What next-token prediction does and does not explain
Trace a small probability table to understand autoregressive language models, sequence likelihood, decoding, and why fluent text still needs evidence.

A language model trained for next-token prediction estimates how likely each possible next token is, given the tokens already present. Repeating that operation produces a sequence. This explains how an autoregressive model generates text; it does not establish that the resulting text is true, useful, or authorized to trigger an action.
For a developer moving into AI, the useful starting point is a probability table. You can inspect one without training a model or knowing the details of a GPU. The table also exposes several mistakes that otherwise hide behind a convincing chat interface.
Follow one small distribution
Consider a hypothetical model with a toy vocabulary larger than the four alternatives shown below. After the prefix The parcel is, suppose these four alternatives carry all the probability mass for this step; other vocabulary tokens have probability zero in this illustrative distribution:
| Next token | Probability | Interpretation in this example |
|---|---|---|
| delayed | 0.50 | Most likely next token |
| arriving | 0.30 | Plausible alternative |
| missing | 0.15 | Less likely alternative |
| blue | 0.05 | Grammatically possible, odd here |
The probabilities add to one. A greedy decoder selects delayed. A sampling decoder can select any of these tokens, according to its sampling rule. Neither operation checks a delivery database. The model has predicted text from its context, not inspected the parcel.
Now append delayed to the context and ask for another distribution. Suppose the probability of today is 0.20. The conditional probability of the two-token continuation delayed today is 0.50 × 0.20 = 0.10. Another branch might assign probability 0.80 to tomorrow after arriving, giving arriving tomorrow a probability of 0.30 × 0.80 = 0.24.
A greedy first step selected the branch whose two-token continuation has the lower probability in this example. Greedy decoding optimizes a local choice. It does not generally find the most probable complete sequence. Searching more branches changes the decoding procedure, although the model’s conditional distributions remain the underlying inputs.
The numbers in the figure are illustrative probabilities, not measurements from a named model. The branch calculation assumes that the second-token probabilities are conditional on the first token already chosen. Multiplying two unconditional probabilities would describe a different model.
The training objective and the product objective
During autoregressive training, a model receives token sequences and learns to assign probability to their observed continuations. A common objective sums negative log probabilities of the target tokens. Predicting a target token with probability 0.8 incurs less loss than predicting it with probability 0.1. The GPT-3 paper describes an autoregressive language model and evaluates how that model performs on downstream tasks using examples supplied in context.
A product team usually cares about something more specific: the assistant must return the current delivery state, quote the correct policy, or produce arguments that a tool can validate. Those goals are related to the quality of the language model but require their own checks. A sentence can be a plausible continuation while describing a parcel that never existed.
For the parcel assistant, the application can call a tracking service using an authenticated order identifier. It then supplies the returned state to the model and asks for a concise explanation. The application must still verify that the answer does not introduce an unsupported delivery date. Supplying evidence gives the model useful context; it does not turn every generated statement into a database fact.
Calculate loss without mistaking it for confidence
The two continuations can also be compared using negative log likelihood. Using natural logarithms, the first has loss about 2.303 and the second about 1.427. The calculation below sums token losses, which is equivalent to taking the negative logarithm of their conditional probability product.
from math import log, isclose
paths = {
"delayed today": [0.50, 0.20],
"arriving tomorrow": [0.30, 0.80],
}
losses = {text: -sum(log(p) for p in probabilities)
for text, probabilities in paths.items()}
assert isclose(losses["delayed today"], -log(0.10))
assert losses["arriving tomorrow"] < losses["delayed today"]
Both examples have two tokens. Comparing raw sequence probabilities across different lengths introduces another consideration: every additional conditional factor is at most one, so multiplying more factors can lower the product. A decoding or evaluation method must state how it handles length and termination. This two-step example does not model sequence termination; a complete generation procedure needs a stopping rule.
Do not read 0.24 as a 24 percent chance that the parcel will arrive tomorrow. It is the model’s probability for this token continuation under the hypothetical context. To estimate factual correctness, you need a labeled task, an explicit definition of correctness and evidence that the proposed confidence measure behaves sensibly on that task. Repeating a confident sentence or selecting a more likely wording supplies none of those by itself.
Andrej Karpathy’s minGPT, inspected on September 20, 2026, is a useful educational bridge from this table to code: its introduction describes token indices entering a transformer and a distribution over the next index coming out. Its README also records a January 2023 semi-archived status. Use it to study that interface, not as a claim that its dependencies or training recipe represent a current production system.
What attention contributes
A transformer computes representations of tokens using attention and other operations. Attention allows the representation at a position to depend on relevant earlier positions. In a decoder using a causal mask, a position cannot attend to future tokens that would be unavailable during generation. The transformer paper specifies scaled dot-product attention and the masking used in its decoder.
This matters when you provide a tracking result near the end of a prompt. The model has a mechanism for using that result while predicting its answer. The existence of the mechanism does not prove that the model used the relevant sentence correctly. You need a behavioral test, such as changing only the delivery state and checking whether the answer changes accordingly.
The training objective also does not make the model a simple table of memorized phrases. Parameters can support generalization across patterns, and a model can produce combinations it has not seen verbatim. At the same time, generation is not evidence of an independent fact-checking process. Both points are compatible with next-token prediction.
A useful application check
Build a tiny fixture with two tracking responses. In one, the parcel is delayed and the service provides no estimated date. In the other, the parcel is scheduled for a stated date. Ask the assistant the same question under both conditions. Accept an answer only if it preserves the distinction between a confirmed date and an unknown one.
You can record the expected properties without prescribing exact wording:
{
"case": "delayed_without_estimate",
"tracking_state": "delayed",
"estimated_delivery": null,
"required": ["mentions delay", "does not invent a date"],
"forbidden": ["claims delivery is guaranteed"]
}
This fixture tests a product requirement. It cannot prove that the assistant will behave correctly for every language, long conversation, or tracking-service failure. Expand it when those conditions matter. The first useful result is narrower: a model’s fluent continuation becomes an application answer only after it meets the application’s evidence and behavior requirements.
The grid makes the distinction testable without requiring one exact sentence. Add a third fixture where the tracking service times out. That case should not inherit yesterday’s delivery estimate merely because a previous message contained it. Record the age and source of any retained result, and decide whether the product permits presenting it as historical information.
As an exercise, write one acceptable and one unacceptable answer for each row before calling a model. Then vary the order identifier and the user’s language while preserving the evidence. If the evaluator only looks for the word “delayed,” an answer that says “delayed, but guaranteed tomorrow” could pass incorrectly. Check supported claims and forbidden additions separately, and inspect disagreements manually. This small evaluation covers a delivery-answer contract; it says little about unrelated reasoning tasks or permission to refund an order.