Skip to main content

AI glossary Operations

What is edge AI?

← All glossary terms
In short

Running AI models on devices or local servers close to where data is produced, to cut latency, keep data local and work without constant connectivity.

Explained

Edge AI means running AI inference on or near the device that produces the data, such as a camera, a sensor, a phone, a vehicle or a server in a factory, instead of sending everything to a central cloud. It reduces latency, saves bandwidth, keeps sensitive data local and lets systems keep working when the connection is poor. Models are usually trained in the cloud and then compressed and deployed to the edge. Federated learning extends the idea to training, by learning from data that stays on the devices.

Why run AI at the edge

  • Latency: decisions in milliseconds, for example on a production line or in a vehicle.
  • Privacy and sovereignty: images, audio or personal data do not leave the site.
  • Connectivity: works offline or with intermittent networks.
  • Bandwidth and cost: send results or alerts, not raw video streams.

How it works

A model is trained centrally, then made smaller and faster through quantisation, pruning or distillation so it fits the memory and power of the target hardware. It is packaged for a runtime on the device and deployed, often to many devices at once.

The hard part is operating the fleet: updating models, monitoring their accuracy in the field, and rolling back when something goes wrong. Many architectures are hybrid: a small model at the edge handles most cases and sends doubtful ones to a larger model in the cloud.

Common uses

  • Visual quality inspection in manufacturing.
  • Predictive maintenance from vibration or sound sensors.
  • Driver assistance and robotics.
  • On-device assistants and transcription on phones and laptops.
  • Retail cameras that count or detect events without storing video.
  • Medical devices that analyse signals at the point of care.

Frequently asked questions

What is edge AI in simple terms?

It is AI that runs on the device itself or on a nearby server, instead of in a distant data centre. A camera that detects a defect on its own, without sending the video to the cloud, is an example.

What is the difference between edge AI and cloud AI?

Cloud AI processes data in central data centres, with plenty of compute and easy updates. Edge AI processes it locally, with lower latency and less data movement but tighter limits on model size. Many systems combine both.

Can large language models run at the edge?

Smaller language models, often quantised, can run on laptops, phones and local servers. The largest models still need data-centre hardware, so edge set-ups typically use a small model locally and call a larger one only when needed.

What is federated learning?

It is a way of training a shared model across many devices without collecting their raw data. Each device trains locally and sends only model updates, which a server combines. It was introduced for mobile devices and fits well with edge AI.

What are the challenges of edge AI?

Limited compute and memory, many different devices to support, updating and monitoring models across a fleet, and physical security of the hardware. Operations tend to be harder than the first deployment.

Decision guides that use this term

How to put it to work

Where it shows up at Thinkia

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert