Skip to main content
Contact

AI glossary Models & Architecture

What is a vision-language model (VLM)?

VLM (Vision-Language Model)

← All glossary terms
In short

A model that understands images and text together, so it can describe pictures, read documents and answer questions about what it sees.

Explained

A vision-language model (VLM) is an AI model that takes images and text as input and reasons across both. It typically combines an image encoder, which turns pixels into numerical representations, with a language model that turns those representations and the user's question into an answer. This lets one model caption photos, read scanned documents, interpret charts and slides, or answer questions about a screenshot. In enterprise use, VLMs power document AI, visual inspection and multimodal assistants, and they extend RAG to content that is not plain text.

How a VLM works

  • An image encoder splits the image into patches and turns them into vectors.
  • A connector maps those vectors into the same space as the language model's tokens.
  • The language model reads the image tokens together with the text prompt and generates the answer.
  • Training on large sets of paired images and text teaches the model to link what it sees with words.

What businesses use VLMs for

  • Document processing: invoices, forms, contracts and scanned files, including tables and handwriting.
  • Visual inspection: spotting defects, damage or missing items in photos from operations.
  • Multimodal search and RAG: answering questions that depend on charts, diagrams or slides, not just text.
  • Customer service: understanding photos that customers send, such as a damaged product or an error screen.
  • Accessibility: describing images for people who cannot see them.

Limits to keep in mind

VLMs can misread small text, dense tables or exact numbers in charts, and they can hallucinate details that are not in the image. Results depend on image quality and resolution.

For critical data, such as amounts on an invoice, it is good practice to validate extracted values against rules or other systems, and to keep a person in the loop for low-confidence cases.

Frequently asked questions

What is the difference between a VLM and an LLM?

An LLM (large language model) works with text only. A VLM adds the ability to take images as input and reason about them together with text. Many current frontier models are multimodal and can do both.

What is an example of a vision-language model?

General-purpose assistants that accept images, such as Claude, GPT or Gemini models, work as vision-language models. There are also open models built for vision tasks that companies can run in their own environment.

Can a VLM replace OCR?

For many documents, yes: a VLM reads the text and also understands layout, tables and context, which classic OCR does not. For very high volumes or strict accuracy on fixed formats, dedicated OCR or a combination of both can still be more reliable and cheaper.

Is a VLM the same as multimodal AI?

A VLM is one type of multimodal AI, focused on images and text. Multimodal AI is the broader term and can include audio, video and other kinds of data.

How are VLMs used with RAG?

They let a RAG system use content that text extraction would lose, such as charts, diagrams and scanned pages. Images can be described or embedded at ingestion, and the VLM can read the retrieved images directly when it answers.

Decision guides that use this term

Where it shows up at Thinkia

Products
The Mesh

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert