Safety Evaluations

Safety evaluations test whether a model or system exhibits defined harmful behaviors under specified conditions. Useful evaluations connect a failure family to a realistic consequence.

Safety Evaluation

Assemble a test suite around the deployed system

Toxicity prompts18% of the current risk mapDeception scenarios27% of the current risk mapTool-use tests31% of the current risk mapDistribution-shift cases24% of the current risk map

Mapped failure coverage18%

A benchmark score describes the cases inside the benchmark. A coverage map keeps the untested failure families visible.

An evaluation plan should name the system boundary, user population, access level, test cases, metric, threshold, and owner. Model-level tests cover behaviors such as toxicity, deception, power seeking, hallucination, or hazardous capability. Application-level tests cover tool calls, data access, workflow errors, and escalation.

Benchmarks can become stale or contaminated when models train on public test data. Refresh cases, preserve hidden sets, and include domain experts who understand what a serious failure looks like.

⚠

Passing Is Scoped

“Passed the safety evaluation” should always be followed by the evaluation name, model and system version, test date, access assumptions, and threshold. Remove that context and the statement becomes marketing.

Checkpoint

Why can a strong model-level safety score fail to predict application safety?

←PreviousCooperative TrainingLearning to Generalize Beyond Training Next→Human Values VerificationAlignment Assurance