Explain overfitting and recognise ways to deal with it

An overfitted model has learned noise specific to its training data, so it does very well in sample and poorly out of sample. Total error splits into bias error from underfitting, variance error from overfitting, and irreducible base error. Adding complexity lowers bias error but raises variance error, and the best model sits where validation error is lowest. The signature of overfitting is low training error with much higher validation error; underfitting shows high error on both. Remedies are reducing complexity, for example with penalised regression or pruning, and choosing how far to go by estimating out-of-sample error with k-fold cross-validation, which rotates each of k parts of the data through the validation role.