AI glossary Models & Architecture
What are parameters in an AI model?
← All glossary termsA parameter is a numerical value a model learns during training; large language models have billions of them, and their count gives the model's size.
Explained
Parameters are the numerical values, mainly the weights of the connections in a neural network, that a model adjusts during training to fit its data. Together they encode what the model has learnt, and once training ends they stay fixed unless the model is fine-tuned. Large language models have billions of parameters, and the size labels in model names, such as 7B or 70B, refer to that count. More parameters generally allow more capability, but they also require more memory and compute to run.
How parameters work
A neural network is made of layers of simple operations. Each connection has a weight, and each unit may have a bias; these are the parameters. During training, the model sees examples, measures its error and adjusts every parameter slightly to reduce it. Repeated over large amounts of data, these adjustments make the network capture patterns in language, images or other data.
Parameters are different from hyperparameters, which are the settings people choose before training, such as the learning rate or the number of layers.
Parameters, size and cost
Parameter count is a rough guide to a model's size. It drives how much memory is needed to load the model and how much compute each answer takes, which affects latency and cost. Research on scaling laws showed that, in language models, quality improves predictably as parameters, data and compute grow together.
Count is not the only factor, though. Training data, training method and architecture matter too. Mixture-of-experts models, for example, have many parameters but only use part of them for each token. Smaller models trained well can match larger ones on specific tasks.
Why it matters for companies
- Model choice: a smaller model can be cheaper and faster for a narrow task.
- Deployment: the parameter count largely sets the hardware needed to run a model on premises or on a device.
- Techniques such as quantisation store parameters with fewer bits to reduce memory and cost, with some loss of quality to test.
- Fine-tuning methods can adjust only a small share of parameters instead of all of them.
Frequently asked questions
What does 7B or 70B mean in an AI model?
It is the number of parameters: 7B means about seven billion and 70B about seventy billion. It is a rough indicator of the model's size, memory needs and running cost.
Are more parameters always better?
No. More parameters usually allow more capability, but data quality, training and architecture also matter. For a specific task, a smaller, well-trained or fine-tuned model can perform as well and cost less.
What is the difference between parameters and hyperparameters?
Parameters are learnt by the model during training, such as the network's weights. Hyperparameters are chosen by people before training, such as the learning rate, batch size or number of layers.
How many parameters does GPT-3 have?
GPT-3, described by OpenAI in the 2020 paper Language Models are Few-Shot Learners, has 175 billion parameters. Many later models do not publish their parameter count.
Do parameters change when I use a model?
No. When you use a model, its parameters stay fixed; it only computes an answer from them. They change only if the model is trained or fine-tuned again.
Where it shows up at Thinkia
Further reading
Related terms
Sources
- Kaplan et al. · Scaling laws for neural language models (arXiv)
- Brown et al. · Language models are few-shot learners (arXiv)
- Goodfellow, Bengio and Courville · Deep Learning (MIT Press)
Updated: 2026-10-21