Chapter 2

Learning Human Preferences

You will be able to explain reinforcement learning, compare forms of human and AI feedback, step through RLHF, and evaluate constitutional approaches to preference learning.

• 1. Introduction to Reinforcement Learning→
• 2. Human Feedback→
• 3. Reinforcement Learning from Human Feedback (RLHF)→
• 4. Reinforcement Learning from AI Feedback (RLAIF)→
• 5. Constitutional AI→