A token is a piece of text — often a word fragment — that language models use as their basic unit of input and output. It is useful for understanding context window limits, pricing, and why long documents must be split or summarized.
Whilst “word count” feels intuitive, in practice token counts diverge from words and blow up with code or odd formatting. For example, a short-looking PDF paste still overflows the context window. We often recommend designing chunk sizes and prompts against token budgets, and watching Time to First Token when streaming latency matters.