Explain how big data and data science are used in investment management
The idea
Text analytics and natural language processing turn filings, transcripts and news into variables a model can use, and sentiment measures are built out of them. Robo-advisers deliver allocation and rebalancing at a cost that makes small accounts economic. Risk analytics aggregate exposures across a whole firm faster than a manual process could, and algorithmic trading works a large order in pieces to limit the price it moves against itself. Before any of that the data has to be captured, curated, stored, searched and moved, and that pipeline is most of the actual work. The limitation to hold on to is that a statistically real pattern in a very large dataset is not the same thing as a tradable signal: transaction costs, the capacity of the strategy, and the speed with which a known pattern is competed away all sit between the two.
A practice question
What does natural language processing let an analyst do?
- Turn text into variables a model can use — correct
- Translate filings into other languagesWhy not: A genuine use of the technology, and not the one that matters to an investment process.
- Write research reports automaticallyWhy not: Generating text is a different task from extracting structure from text somebody else wrote.
Filings, transcripts and news are written for people and cannot be analysed as they stand. Processing them into counts, scores and categories is what makes them available to anything quantitative.
A harder one
Sentiment extracted from earnings call transcripts is most useful when it does what?
- Captures something the reported numbers do not — correct
- Confirms what the reported figures already showWhy not: A measure that agrees with the figures adds nothing that was not already available to everyone.
- Predicts the share price directlyWhy not: Almost nothing predicts price directly, and a measure claiming to is the one to distrust most.
Hesitation, hedging and a change in how management talks about a segment can carry information beyond the reported numbers. A sentiment measure is most useful when it adds incremental information not already reflected in the accounts, rather than restating them.
The hardest tier
A backtest over a very large dataset finds a highly significant pattern. What stands between that and a strategy?
- Costs, capacity, and how fast it is competed away — correct
- Nothing, provided the significance is high enoughWhy not: Significance answers whether a pattern is real. It never answers whether it can be traded at a profit.
- A larger sampleWhy not: More data sharpens the estimate and does nothing about what it costs to act on it.
A real pattern can still be unprofitable after transaction costs, too small to absorb meaningful capital, or already known and arbitraged. Each of the three is a separate hurdle, and a significant backtest clears none of them.
In the app: 9 practice questions on this outcome, a narrated walkthrough with diagrams, spaced review that brings it back before you forget it.
Take the free diagnostic →More in Introduction to Financial Data Science
- Explain which parts of fintech matter for collecting and analysing financial data
- Explain what is meant by machine learning, artificial intelligence and big data
- Explain deep, supervised and unsupervised learning and how investment managers use them
- Explain how training works for large language models and the way they produce text
- Explain how investment managers put large language models and generative AI to use
- Explain where AI and large language models fall short or pose risks in investment work