Moral AI
An AI system allocates scarce medical resources during a mass-casualty event. Should it prioritize survival probability, number of lives saved, years of life, urgency, or equal chance? The system needs a policy before the emergency.
Moral AI studies systems that reason or act using ethical principles and values.
Moral AI
Choose how a medical allocation policy learns
Rules set boundaries; examples guide decisions inside them.
A top-down approach encodes principles or rules. It is easier to inspect and struggles with exceptions. A bottom-up approach learns from examples. It adapts to context and can reproduce historical injustice. Hybrid approaches use rules to set boundaries and data to guide choices inside them.
The model's prediction adds another moral layer. A survival estimate can be uncertain or biased across groups. A policy that sounds ethical can produce unethical outcomes when its factual inputs are unreliable.
Decide Before the Car Must Decide
Autonomous systems make high-stakes choices too quickly for case-by-case deliberation. Designers need policies, uncertainty handling, and fail-safe behavior before deployment.
Which part of a medical allocation system deserves verification before debating the allocation rule itself?