In simple words: Tokens are text pieces; a vocabulary maps them to numbers.
Split the text
Tokenization divides text into tokens, which may be words, parts of words, or punctuation. The exact tokens depend on the tokenizer. A vocabulary is the list of tokens the model recognises and assigns number IDs.
Why numbers are needed
Neural networks calculate with numbers. A token ID identifies a piece of text, but an ID alone does not describe its meaning. Embeddings add a more useful numeric representation. An unknown or rare word may be split into smaller pieces.
See the flow
sentence→tokens→token IDs→model
Simple example
“unhappiness” might become one token or several smaller tokens. The model converts those tokens to IDs before using them.
Remember: Tokens are text pieces; a vocabulary maps them to numbers.