Skip to content
Instant download · PDF + web edition · Free sample of every book · 30-day refund promise
Latent Library Browse books
Fundamentals

Reinforcement Learning Explained

Learning From Rewards, Step by Step

Pages
81
Chapters
59
Level
Intermediate
Format
E-book

About this book

Reinforcement learning taught computers to play games at superhuman level and now helps shape the behaviour of language models. This book explains how it works, clearly and with worked numbers.

You will start with the exploration–exploitation dilemma in multi-armed bandits, then learn Markov decision processes, value functions, dynamic programming, Monte Carlo and temporal-difference learning, Q-learning, deep Q-networks and policy-gradient methods. Final chapters cover reinforcement learning from human feedback, safety concerns such as reward hacking, and when RL is the wrong tool.

Small grid-world examples make every algorithm concrete.

What you will learn

  • Exploration versus exploitation with bandits
  • States, actions, rewards and value functions
  • Dynamic programming, Monte Carlo and TD learning
  • Q-learning and deep Q-networks
  • Policy gradients and actor–critic methods
  • RL from human feedback, reward hacking and safety

Table of contents

  1. IntroductionIntroduction
  2. Chapter 1Learning From Rewards
  3. Chapter 2How RL Differs From Other Learning
  4. Chapter 3The Multi-Armed Bandit
  5. Chapter 4Estimating Action Values
  6. Chapter 5Exploration Strategies
  7. Chapter 6Regret
  8. Chapter 7Contextual Bandits
  9. Chapter 8Designing Rewards
  10. Chapter 9Markov Decision Processes
  11. Chapter 10A Grid World
  12. Chapter 11Returns and Discounting
  13. Chapter 12Policies
  14. Chapter 13Value Functions
  15. Chapter 14The Bellman Equation
  16. Chapter 15Optimal Values
  17. Chapter 16Dynamic Programming
  18. Chapter 17When the Model Is Unknown
  19. Chapter 18Monte Carlo Learning
  20. Chapter 19Temporal-Difference Learning
  21. Chapter 20SARSA
  22. Chapter 21Q-Learning
  23. Chapter 22On-Policy and Off-Policy Learning
  24. Chapter 23n-Step Methods and Eligibility Traces
  25. Chapter 24Function Approximation
  26. Chapter 25The Deadly Triad
  27. Chapter 26Model-Based RL and Planning
  28. Chapter 27Search and Learning Together
  29. Chapter 28Deep Q-Networks
  30. Chapter 29Policy Gradient Methods
  31. Chapter 30Baselines and Advantage
  32. Chapter 31Actor–Critic Methods
  33. Chapter 32Trust Regions and PPO
  34. Chapter 33Continuous Control
  35. Chapter 34Simulation and Sim-to-Real
  36. Chapter 35Offline Reinforcement Learning
  37. Chapter 36Imitation Learning
  38. Chapter 37Reinforcement Learning From Human Feedback
  39. Chapter 38Reward Hacking
  40. Chapter 39Safe Exploration
  41. Chapter 40Applications in Practice
  42. Chapter 41When RL Is the Wrong Tool
  43. Chapter 42Practical Tips
  44. Chapter 43A Worked Q-Learning Episode
  45. Chapter 44Q-Learning in Code
  46. Chapter 45Evaluating RL Agents
  47. Chapter 46Value Iteration, Worked Through
  48. Chapter 47Reward Shaping
  49. Chapter 48Partial Observability
  50. Chapter 49Hierarchical Reinforcement Learning
  51. Chapter 50Multi-Agent Reinforcement Learning
  52. Chapter 51Curiosity and Intrinsic Motivation
  53. Chapter 52Rewards From Verifiable Outcomes
  54. Chapter 53Hyperparameters That Matter
  55. Chapter 54Debugging RL
  56. Chapter 55RL for Long-Term Recommendations
  57. Chapter 56The Ethics of Optimising Behaviour
  58. Chapter 57Case Study: Energy Management
  59. Chapter 58Case Study: Adaptive Website Experiments
  60. Chapter 59A Project Checklist
  61. AppendixExercises
  62. GlossaryGlossary

Who this book is for

Students and developers with basic algebra and some programming experience who want to understand reinforcement learning.

Details

FormatPDF + web edition
Length81 pages (approx.)
Chapters59
LevelIntermediate
LanguageEnglish
TopicFundamentals
AuthorLatent Library Editorial

Keep reading

You may also like

All books
$12.99
Details