How-to · ORI · ORI.1

Token — one definition with a model

Tokens are pieces models process — not always whole words. Use that when length/limits confuse you.

Open any chat tool and paste: 'Explain tokens vs words for a beginner in 3 short bullets with one example of a word that might split into multiple tokens.' You're done with 3 bullets plus one example of a word getting split.

Worked example

A token is just a chunk of text the model reads — often a piece of a word, not always a whole one. And when you hear about limits, they're usually measured in tokens, not pages.

← All How-tos