How does an AI model pick its next word?

A language model reads and writes tokens: chunks of text from a fixed list tens of thousands long. Common words are one token; rarer words are split into pieces, so the model never sees individual letters.

To write the next token the model gives every token in its vocabulary a score, computed from patterns that training set into billions of numbers. It looks nothing up: a high score for ' Paris' comes from the pattern, not a stored fact.

The scores are turned into chances that add up to 100% by raising e to each score and dividing by the total (softmax). A lead in score becomes a much bigger lead in chance: scores of 3, 1 and 0 give the top token 84%, not 75%.

At normal settings the model doesn't always take the top token: it draws one at random, weighted by the chances, like a loaded die. That is why the same prompt can get different replies.

Temperature divides every score before the chances are worked out. Below 1 the gaps widen and the top token wins almost every time; above 1 they shrink and unlikely tokens get picked more. It changes how adventurous the picking is, not what the model knows.

The model writes one token, adds it to the text, and runs again on the longer text to choose the next. There is no finished draft it copies out: each token is picked only after the ones before it exist, so what it has already written steers what comes next.