It sounds like this“Turn the temperature up and the model thinks harder and knows more.”
Thinking temperature changes how clever the model is
Why it feels right
'Hot' feels energetic and 'cold' feels rigid, and some tools label the slider 'creativity'. At a high setting the output reads livelier and less predictable, and we tend to take variety for imagination. But the model has already finished its work by the time temperature acts. The scores are done; temperature only changes how they are turned into chances.
Why it’s wrong
The scores come out the same at any temperature; temperature only reshapes the chances drawn from them. Higher means more varied and more often wrong, lower means more predictable.
A worked example
Three candidate next words have scores of 3, 1 and 0. At temperature 1 their chances are 84%, 11% and 4%.
At temperature 0.5 every score is divided by 0.5, so they become 6, 2 and 0. The top word now gets about 98%, and the others about 2% and 0.2%. The model will pick the same word almost every time.
At temperature 2 the scores become 1.5, 0.5 and 0, and the chances flatten to about 63%, 23% and 14%. Now the 4% word is picked one time in seven.
The model's scores, which are everything it knows, were the same in all three cases. For a question with one right answer, a higher temperature just means more draws that land on the wrong one.
How to catch yourself
- You raise the temperature to get a smarter or more thoughtful answer to a factual question.
- You set it to zero hoping for the best possible answer.
- You think of temperature as how hard the model is working.
What to hold instead
Temperature reshapes the chances without changing the scores. Lower is more predictable and repeatable, which suits facts, code and extracting information. Higher is more varied, which suits brainstorming and creative drafts, at the price of more mistakes. It does not change what the model knows or how well it reasons.
Where you’ll meet it
- API settings and 'creativity' sliders in writing tools.
- Choosing settings for code generation, where a lower temperature makes output steadier.
- Brainstorming, where several varied attempts are wanted.
Questions people ask
- What temperature should I use?
- Low (0 to 0.3) for facts, code and anything with one right answer. Medium (0.7 to 1) for general writing. Higher for brainstorming where you want many different options and will pick the good ones.
- What does temperature 0 do?
- It makes the model always take the highest-scoring token, so replies are nearly the same each time. It does not make the model smarter, and a predictable answer can still be wrong.