Introduction to Reinforcement Learning

Suppose a cleaning robot sees dirt one tile to its right. It can move left, move right, or clean its current tile. Nobody labels the correct action for every possible room. Instead, the robot tries actions and receives consequences.

Reinforcement learning trains an agent to choose actions in an environment to maximize cumulative reward.

The RL Loop, Step by Step

Guide the mouse to the cheese

State (s)

The mouse's current position in the maze — what it "knows" about where it is.

Action (a)

Possible moves from that position. Example: up, down, left, right.

Reward (r)

🧀 Cheese = +10
🚫 Dead end = −1
🐾 Normal move = 0

Policy (π)

The mouse's strategy — a rule mapping each position (state) to a move (action).
Example: "If I smell cheese, go forward; otherwise, turn at dead ends."

🐭

🧀

Steps taken

Cumulative reward

Last outcome

—

Every move is a state-action-reward cycle. The policy is whatever rule you were using in your head to decide each move — reinforcement learning is the process of improving that rule from experience.

Supervised Learning vs. Reinforcement Learning

Click a part of either diagram to see what it means

Supervised Learning

Solving prediction problems by training a model on a fixed dataset using ground truth labels (or proxy targets) to compute a loss and update parameters so the model maps I/O accurately.

Reinforcement Learning

Solving decision problems by building agents that learn from the environment by interacting with it through trial and error and receiving rewards (+ & -) as feedback.

Agent

The decision-maker. It observes a state and selects an action according to its policy — the RL analogue of the model.

Supervised learning fits predictions to fixed labels. Reinforcement learning fits behavior to consequences the agent's own actions help produce, so the reward supplies the direction and reward design becomes behavior design.

A state describes what the agent can observe. An action changes the environment. A policy maps states to actions. A reward evaluates an immediate outcome. The return combines rewards across time.

The update in the lab moves the estimated value of an action toward a target: immediate reward plus the discounted value of the next state. The discount factor exists because future reward matters, while distant estimates carry more uncertainty.

This video is a great foundation to beginning to understand RL.

◆

Why Alignment Enters Immediately

The policy learns what the reward reinforces. If cleaning a mess earns praise and making a mess carries no cost, a child, animal, or robot can discover the same profitable loop: create a mess, then clean it.

Duke University (AIPI 590)

Introduction to Modern Reinforcement Learning

A hands-on course bridging foundational RL theory with state-of-the-art methods, from classical tabular methods through deep RL architectures and human-in-the-loop learning.

Duke University (AIPI 590)View the course

Checkpoint

What does a policy represent?

←PreviousMisalignment Failure ModesThe Alignment Problem Next→Human FeedbackLearning Human Preferences