It sounds like this“It reads what I type letter by letter, so of course it can spell.”
Picturing the model reading letter by letter, as we do
Why it feels right
We read in letters, and the model's replies look like they came from someone who read ours. It writes essays, fixes spelling, even plays word games, so it seems obvious that it sees the letters in what you type. But before the model gets your text, a separate step cuts it into chunks called tokens and swaps each chunk for a number. From then on, the letters inside a chunk are gone.
Why it’s wrong
It reads tokens, chunks of text, and the letters inside a chunk never reach it. That is why it can write fluently and still miscount the r's in 'strawberry'.
A worked example
Take 'strawberry'. A typical tokeniser might cut it into 'str', 'aw' and 'berry'. The exact cuts vary from model to model, but the point is the same: the model receives three chunks, not ten letters.
Asked how many r's are in the word, it has to answer from what it has learned about those chunks. It has seen text that spells words out and text that counts letters, so it often gets this right, and often does not. What it has never done is look at the letters and count them. That is why a model can write a fluent essay about strawberries and then say the word has two r's.
The same gap explains why models stumble on reversing a word, finding words that start and end with the same letter, or writing a sentence with no letter 'e'.
How to catch yourself
- You expect a model that writes beautifully to be flawless at spelling, counting letters or reversing words.
- You assume it reads a typo as a typo, the way you do.
- You are surprised that it fails at word-game tasks that a child could do.
What to hold instead
The model reads tokens, and what is inside a token is something it only knows from patterns in text. For letter-level tasks, work the way you would with someone who has only ever heard words spoken: ask it to spell the word out one letter at a time first, and count from that list. Once the letters are written down, they are separate tokens it can read.
Where you’ll meet it
- Counting letters in a word, or words in a sentence.
- Word games like Wordle, anagrams and crosswords.
- Reversing a string, or formatting text letter by letter.
Questions people ask
- Why can't ChatGPT count the letters in a word?
- Because it does not see letters. Text is split into tokens, chunks of a few characters, before the model reads it. It can only guess at what is inside a chunk from patterns it learned in training, so letter counting is unreliable.
- What is a token?
- A chunk of text from a fixed list the model uses, usually tens of thousands of entries long. In English a token averages about four characters, so a common word is often one token and a rare word is several.