← GlossaryTechnical

What is Tokens?

The basic units of text processed by an LLM, representing sequences of characters, words, or sub-words.

More terms