AI Safety, Alignment and Evaluation

Every AI company has to answer two questions about its product: what can it do, and what will it do? The first is a question of capability; the second is a question of behaviour, which includes doing what the user and the company intend (alignment), refusing what it should refuse, telling the truth, not causing harm when misused, and not taking unintended actions as an agent. Evaluation is how both questions are answered: benchmarks, red-teaming, dangerous-capability tests, product-specific test sets and monitoring in production. Evaluation is also a commercial instrument. Benchmark scores sell models, safety commitments win enterprise and government trust, and the absence of evidence about behaviour becomes a liability when something goes wrong, as lawsuits over chatbot interactions with teenagers showed in 2024–26. The field's central difficulty is that the evidence is weaker than the claims: public benchmarks saturate within a year or two, leak into training data, and reward optimising for the test; red-teaming finds problems but cannot prove their absence; and frontier labs write, grade and revise their own safety frameworks, with legal transparency duties only beginning in 2025–26 (the EU AI Act's general-purpose model rules, California's SB 53, New York's RAISE Act). This section explains the purposes and methods of evaluation, the limits of benchmarks, how alignment is done and where it fails, red-teaming and dangerous-capability testing, the labs' frontier safety frameworks, the emerging legal regime, the incidents that shaped the field, the business of safety, landmark cases, pitfalls and how practitioners think about it. Regulation in general is in 19; agent reliability is in 13.