What Is a Token? (And Why AI Tools Charge for Them)

The short answer: A token is the small chunk of text an AI model actually reads and generates — roughly a word or part of a word. AI tools measure their memory limits and usage costs in tokens, which is why long conversations eventually get cut off and why AI pricing is quoted “per token” rather than “per word.”

If you’ve ever used an AI tool’s API, hit a “conversation too long” message, or glanced at a pricing page, you’ve run into the word “token” without necessarily knowing what it means. It’s one of the more useful pieces of AI vocabulary to understand, because it explains several behaviours that otherwise seem confusing.

So a token isn’t quite a word

AI language models don’t process text word by word — they process it token by token, where a token is a chunk of text that’s sometimes a whole word, sometimes part of a word, and sometimes a single punctuation mark. Common short words (“the,” “cat,” “run”) are usually a single token. Longer or less common words (“unbelievable,” “tokenization” itself) often get split into two or three tokens. As a rough rule of thumb, a sentence of plain English averages around three-quarters of a token per word — so 100 words is roughly 130–150 tokens.

Why AI models think in tokens, not words

Splitting text into tokens (a process called tokenization) lets a model handle any text — including made-up words, typos, or text in other languages — using a manageable, fixed vocabulary of token pieces, rather than needing a separate entry for every possible word. It’s a technical convenience for how the model is built, but it has very real, visible effects on how AI tools behave.

Tokens and the “context window”

Every AI model has a maximum number of tokens it can take into account at once — your prompt, any documents you’ve attached, the conversation history so far, and the response it’s about to generate, all combined. This limit is called the context window. A model with a 128,000-token context window can “see” roughly 100,000 words at once (prompt plus response combined); beyond that, something has to be dropped.

This is directly related to multimodal AI capabilities too — images, audio, and video all get converted into token-equivalent units that count against the same limit, which is part of why uploading a large file or long video can use up a lot of your available context very quickly.

Tokens and pricing

AI providers that charge by usage (rather than a flat monthly subscription) typically price by the token — a certain cost per million input tokens and a separate, usually higher, cost per million output tokens. This is why a short question with a short answer costs a fraction of a cent, while feeding a long document into an AI tool and asking for a detailed analysis costs measurably more. If you’ve ever wondered why a subscription like ChatGPT Plus or Claude Pro (see the subscription comparison) is priced as a flat monthly fee instead, it’s because the provider is absorbing token costs across all your usage rather than billing per request.

Why your AI tool sometimes “forgets” earlier parts of a chat

When a conversation runs long enough to approach the model’s context window limit, something has to give. Most consumer AI tools handle this by quietly dropping, summarising, or de-prioritising the earliest messages to make room for new ones. That’s the mechanism behind the common experience of an AI assistant “forgetting” something you mentioned much earlier in a long chat — it’s not really forgetting in a human sense, it’s a hard limit on how many tokens it can hold in view at once.

Do you need to think about this day to day?

Not really, for typical use. If you’re having a normal conversation with an AI tool through a chat interface, you’ll rarely hit token limits. It becomes relevant if you’re working with very long documents, very long conversations, or using AI through a pay-per-use API rather than a flat subscription — in those cases, understanding tokens explains both the cost and the behaviour you’re seeing.

Frequently asked questions

What is a token in AI? A token is the small chunk of text an AI model actually reads and generates — roughly a word, part of a word, or a punctuation mark. A short word like “cat” might be one token, while a longer or less common word might be split into two or three tokens. AI models process and measure text in tokens rather than in words or characters.

Why do AI tools charge per token? Processing each token costs computing resources, so AI providers charge based on how many tokens are in your input plus how many are generated in the output. This is why a long, detailed prompt with a long response costs more than a short question with a short answer.

What is a context window? A context window is the maximum number of tokens an AI model can take into account at once — covering your prompt, any attached documents, and the conversation history, plus the response it generates. Once a conversation exceeds that limit, the model starts losing track of the earliest parts.

Why does ChatGPT forget things I said earlier in a long conversation? Because the conversation has exceeded the model’s context window in tokens. When that happens, the system typically drops or summarises the oldest messages to make room for new ones, which is why a very long chat can feel like the AI has “forgotten” something you mentioned much earlier.