Tokenisation

Splitting text into units, usually words, called tokens: the first step in preparing text for a model. With digital assets the same word means representing ownership of an asset as a token on a ledger.

Where you will meet it

Tokenisation appears in 2 learning outcomes on the CFA syllabus. A few of them:

• Explain distributed ledger technology and where it is used
  CFA Level I · Alternative Investments

  “The credible applications in finance are settlement, tokenisation of illiquid assets and smart contracts that execute automatically.”

• Explain techniques for extracting, choosing and engineering features from text data
  CFA Level II · Quantitative Methods

  “Tokenisation splits it into words; normalisation lowercases, removes stop words, and applies stemming or lemmatisation.”

See also

Token

Practise it

• Today's practice question One question, with every wrong option explained.
• Free Level I diagnostic test Twelve questions that show which topics to start with.
• Free finance calculators Check your arithmetic on the exam's core formulas.

All terms beginning with T · Glossary