Cooperative Training

Most deployed systems share an environment with people, services, and other agents. A policy trained alone can fail when success depends on coordination.

Multiple agents play hide and seek trained through RL.

In the stag hunt, two agents can choose a high-value shared task or a lower-value individual task. The shared task pays more only when both coordinate.

Cooperative Training

Play a stag hunt with an unfamiliar agent

Expected shared-task payoff

2.68

Safe individual payoff

2.00

Policy choice: coordinate on the shared task

Cooperation depends on beliefs about other agents, communication, and payoff design. High shared reward alone may fail to produce coordination.

Cooperative training exposes agents to joint decisions, mixed incentives, and unfamiliar partners. Fully cooperative settings share reward. Mixed-motive settings combine shared and individual goals. Zero-shot coordination tests whether a policy can work with partners it never met during training.

Real applications include bandwidth allocation, warehouse fleets, traffic control, and assistants that hand work between specialized agents. Coordination failures can waste resources even when each agent appears locally competent.

Checkpoint

Why might an agent choose the lower-value individual task in a stag hunt?

←PreviousAdversarial TrainingLearning to Generalize Beyond TrainingNext→Safety EvaluationsAlignment Assurance