LATENT LIBRARY AI ENGINEERING Small Modelsand On-DeviceAI Fast, Private and Affordable AI atthe Edge LATENT LIBRARY EDITORIAL

Small Models and On-Device AI

Fast, Private and Affordable AI at the Edge

Latent Library Editorial

IntroductionIntroduction

For years, AI meant sending data to the cloud and waiting for an answer. That is changing. Phones, laptops, cars, cameras, watches and industrial sensors now contain dedicated AI accelerators, and models have become small and efficient enough to run on them. Speech recognition, photo enhancement, text prediction, translation, and even compact language models increasingly run directly on the device.

On-device AI offers privacy (data stays local), speed (no network round trip), offline operation and lower cloud costs. It also brings constraints: limited memory, compute, battery and storage, a huge variety of hardware, and the challenge of updating models across millions of devices.

This book is a practical guide for product managers, engineers and technical leaders deciding when and how to run AI on devices. It covers hardware, model compression, runtimes, application design, privacy, security, testing and operations, with worked examples.

How the book is organised

Chapter 1What Runs Where

AI workloads can run:

Many products combine them.

You have reached the end of the free sample.
The full book continues with 71 more chapters, plus the appendices.

Get the full book