Skip to main content
Contact

AI glossary Models & Architecture

What is overfitting in machine learning?

← All glossary terms
In short

When a model learns its training data too closely, including the noise, and then performs poorly on new data it has not seen.

Explained

Overfitting happens when a model fits its training data so closely that it memorises details and noise instead of learning the general pattern. It scores well on the data it was trained on but makes poor predictions on new data. The usual sign is a gap that opens between performance on training data and on a separate validation set. Common causes are a model that is too complex for the amount of data, too little or unrepresentative data, and training for too long. Regularisation, more and better data, early stopping and proper validation help prevent it.

How to detect it

Keep data the model never sees during training: a validation set to tune it and a test set for the final check. Track the error on both as training progresses.

If training error keeps falling while validation error stops improving or starts rising, the model is overfitting. A model that looks excellent in development and disappoints in production is often a case of overfitting, or of training data that does not represent real use.

Overfitting vs underfitting

  • Overfitting: the model is too closely tuned to the training data. Low training error, high error on new data.
  • Underfitting: the model is too simple to capture the pattern. High error on both.
  • Good fit: the model captures the pattern and generalises, with similar error on training and new data.

How to prevent it

  • Use more data, and data that reflects the real cases the model will face.
  • Choose a simpler model, or constrain a complex one with regularisation.
  • Use dropout in neural networks, which randomly disables units during training.
  • Stop training early when validation performance stops improving.
  • Use cross-validation to check results are stable across data splits.
  • In fine-tuning large models, keep evaluation sets separate and watch for the model memorising examples.

Frequently asked questions

What is overfitting in simple terms?

It is when a model learns the examples by heart instead of learning the rule behind them. It does very well on the data it studied and badly on anything new, like a student who memorises past exam answers without understanding the subject.

How do you know if a model is overfitting?

Compare its performance on training data with its performance on a validation set it has not seen. A large gap, with much better results on training data, is the clearest sign.

What is the difference between overfitting and underfitting?

Overfitting means the model is too closely tied to the training data and fails on new data. Underfitting means it is too simple and performs poorly even on training data. The aim is a balance between the two.

Can large language models overfit?

Yes, especially during fine-tuning on small data sets, where a model can memorise examples and lose general ability. Evaluating on held-out data and keeping test sets out of training data are the main safeguards.

Decision guides that use this term

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert