It sounds like this“It finds the answer in its database, or searches the internet, and types it out.”
Picturing the model looking the answer up
Why it feels right
Search engines and databases taught us that when a computer answers a question, it retrieves something. A chatbot answers questions, and often correctly, so retrieval is the natural picture. But a plain language model has no catalogue to retrieve from. Its knowledge is spread across billions of numbers set during training, and an answer is built, token by token, from that.
Why it’s wrong
There is no database of answers behind a plain model. It computes a score for every possible next token from patterns in its weights, which is also why it can produce a fluent answer that is false: a plausible pattern is not a checked fact.
A worked example
Ask a model for a book about the history of salt and it might answer 'Salt: A World History, by Mark Kurlansky', which is real. Ask for a lesser-known topic and it may write something like 'The Salt Roads of Medieval Europe, by Elena Marsh'. Each word of that title is a likely continuation of the words before it, and each is a sensible choice. At no step does it check a library catalogue, so a title can come out that sounds right and does not exist.
A model that is connected to a search tool behaves differently. The tool fetches pages, and their text is placed in front of the model as part of what it reads. Then the answer can draw on what was fetched. That is retrieval added around the model, and it is why the same model can be more reliable in one product than another.
How to catch yourself
- You read a confident tone as a sign it checked.
- You ask for exact quotes, citations, phone numbers or statistics and trust what comes back.
- You assume it would say so if it did not know.
What to hold instead
Think of a plain model as writing the most plausible continuation, not fetching a record. For common facts, plausible and true overlap a great deal. For rare ones, such as a specific citation, a small detail or something recent, they part ways, and the model has no alarm that goes off when they do. Check anything specific, or use a version connected to search.
Where you’ll meet it
- Invented citations and studies in essays.
- Legal filings that cited cases a chatbot made up, which has led to lawyers being sanctioned.
- Code that calls a function or library option that does not exist.
Questions people ask
- Why does AI make things up?
- Because it generates the most plausible next token, not a checked fact. When a fact is rare or missing from what it learned, the plausible-sounding continuation is still produced. This is often called hallucination.
- Does ChatGPT search the internet?
- Only when a search tool is switched on. Without it, the model answers from patterns in its training. With it, pages are fetched and placed in front of the model, which is retrieval added around the model rather than something inside it.