OPEN CURRICULUM
Technical AI Safety Curriculum
Our intro fellowship curriculum is open source. Every week's materials are posted here so anyone can read along, or run their own group.
Week 1
How modern language models work, and why aligning them with what we actually want is hard: specification gaming, reward hacking, and how failures could play out.
- 3Blue1Brown — Large Language Models explained briefly
- Rational Animations — Specification Gaming: How AI Can Turn Your Wishes Against You
- Ajeya Cotra — Why AI Alignment could be hard with modern deep learning
- METR — Recent Reward Hacking
- AI In Context — If you remember one AI disaster, make it this one
- Paul Christiano — What Failure Looks Like
Week 2
Inner alignment and mesa-optimizers, and Constitutional AI as an approach to training models to follow written principles.
Week 3
How models are trained on human feedback, whether that feedback can scale to hard tasks, and evidence of models faking alignment or scheming.