Set out the aims, stages and examples of data preparation and wrangling
The idea
Preparation (cleansing) fixes errors: incompleteness (missing values), invalidity (values outside a meaningful range), inaccuracy (values that are not a true measure), inconsistency (data conflicting with other data), non-uniformity (different formats) and duplication. Wrangling transforms the cleaned data: extraction, aggregation, filtration, selection and conversion of data types. Outliers can be trimmed, removing extreme observations, or winsorised, replacing them with the values at chosen percentiles so none is lost. Scaling puts features on comparable scales: normalisation, (X − min) ÷ (max − min), maps to 0 to 1 but is sensitive to outliers; standardisation, (X − mean) ÷ standard deviation, centres on zero. The two scaling formulas are easy to mix up.
In the app: 3 practice questions on this outcome, a narrated walkthrough with diagrams, spaced review that brings it back before you forget it.
Take the free diagnostic →More in Big Data Projects
- Recognise the stages of a data analysis project and explain each one
- Judge how well a machine learning algorithm fits
- Set out the aims, techniques and examples of exploring data
- Explain techniques for extracting, choosing and engineering features from text data
- Set out the aims, stages and techniques of training a model
- Explain how text-based data is prepared, wrangled and explored for use in financial forecasting