Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 3 - LLM training

For more information about Stanford’s graduate programs, visit: https://online.stanford.edu/graduate-education

To follow along with the course schedule and syllabus, visit: https://cme295.stanford.edu/syllabus/

Chapters:
00:00:00 Introduction
00:03:36 Pretraining
00:10:33 Cross-entropy loss
00:17:15 Data mixtures
00:22:18 Midtraining
00:23:41 Long context training
00:31:47 Supervised finetuning (SFT)
00:40:15 Low-Rank Adaptation (LoRA)
00:46:37 Preference tuning
00:54:44 Bradley-Terry formulation
01:01:49 Reinforcement Learning from Human Feedback (RLHF)
01:05:38 Direct Preference Preference Optimization (DPO)
01:12:08 Reasoning
01:20:56 Group Relative Policy Optimization (GRPO)
01:23:31 On-Policy Distillation (OPD)
01:27:47 End-to-end example

Afshine Amidi is an Adjunct Professor at Stanford University.

Shervine Amidi is an Adjunct Professor at Stanford University.

Follow the playlist: https://youtube.com/playlist?list=PLakGfbKCei80&si=HF2tNywj7nXIQq8X