Small Models and On-Device AI
Fast, Private and Affordable AI at the Edge
- Pages
- 84
- Chapters
- 72
- Level
- Advanced
- Format
- E-book
About this book
Not every AI task needs a giant model in a data centre. Small models running on devices can be faster, cheaper, more private and available offline. This book explains how to get there.
You will learn how to estimate memory and compute budgets, choose small model architectures, and compress models with quantisation, pruning and distillation. Chapters cover runtimes and hardware accelerators, latency and battery considerations, evaluation of compressed models, hybrid designs that combine on-device and cloud models, and updating models in the field.
Worked examples show the arithmetic behind every trade-off.
What you will learn
- Budgeting memory, compute, latency and energy
- Choosing and designing small models
- Quantisation, pruning and distillation
- Runtimes, accelerators and deployment targets
- Evaluating compressed models fairly
- Hybrid on-device and cloud architectures
Table of contents
- IntroductionIntroduction
- Chapter 1What Runs Where
- Chapter 2Why Run on the Device
- Chapter 3Why Not
- Chapter 4Deciding Where to Run
- Chapter 5Hybrid Designs
- Chapter 6Familiar Examples
- Chapter 7Industrial and Embedded Examples
- Chapter 8Tiny Machine Learning
- Chapter 9Small Language Models
- Chapter 10The Business Case
- Chapter 11Device Hardware
- Chapter 12Memory Is the Real Limit
- Chapter 13Battery and Heat
- Chapter 14Hardware Fragmentation
- Chapter 15Why Compress Models
- Chapter 16Quantisation
- Chapter 17Quantisation Methods
- Chapter 18Pruning
- Chapter 19Distillation
- Chapter 20Efficient Architectures
- Chapter 21Task-Specific Models
- Chapter 22Measuring the Accuracy Cost
- Chapter 23Inference Runtimes
- Chapter 24Model Formats and Conversion
- Chapter 25Hardware Delegates
- Chapter 26Running Language Models Locally
- Chapter 27Model Size Budgets
- Chapter 28Shipping Models
- Chapter 29Operating-System Models
- Chapter 30Updating Models
- Chapter 31Benchmarking
- Chapter 32Device Farms
- Chapter 33Browser-Based AI
- Chapter 34Edge Servers
- Chapter 35Designing for Speed
- Chapter 36Designing for Uncertainty
- Chapter 37Offline Experience
- Chapter 38Personalisation On Device
- Chapter 39Federated Learning
- Chapter 40Vision Use Cases
- Chapter 41Audio and Speech Use Cases
- Chapter 42Text Use Cases
- Chapter 43Sensor and Health Use Cases
- Chapter 44Automotive Use Cases
- Chapter 45Industrial Use Cases
- Chapter 46Assistants on Device
- Chapter 47Privacy Benefits
- Chapter 48Privacy Pitfalls
- Chapter 49Model Security
- Chapter 50Adversarial Inputs
- Chapter 51Update Integrity
- Chapter 52Testing On-Device Models
- Chapter 53Monitoring in the Field
- Chapter 54Handling Drift
- Chapter 55Accessibility
- Chapter 56Energy and Sustainability
- Chapter 57Licensing
- Chapter 58Team Skills
- Chapter 59Worked Example: Offline Document Scanner
- Chapter 60Worked Example: Local Writing Assistant
- Chapter 61Worked Example: Factory Defect Detection
- Chapter 62Case Study: Keyboard Prediction
- Chapter 63Case Study: Hearing Assistance
- Chapter 64Case Study: Retail Shelf Monitoring
- Chapter 65Case Study: Field Inspection App
- Chapter 66Common Pitfalls
- Chapter 67Placement Checklist
- Chapter 68Model Readiness Checklist
- Chapter 69Release Checklist
- Chapter 70Template: Model Budget Sheet
- Chapter 71A First-Project Plan
- Chapter 72Looking Ahead
- GlossaryGlossary
Who this book is for
ML and mobile engineers building AI features for phones, laptops, browsers and embedded devices.
Details
| Format | PDF + web edition |
|---|---|
| Length | 84 pages (approx.) |
| Chapters | 72 |
| Level | Advanced |
| Language | English |
| Topic | AI Engineering |
| Author | Latent Library Editorial |
Keep reading