Lectures

You can download the lectures here. 🔔 Subscribe to our newsletter for the latest updates on LLMs!

• 1. Course Introduction
  Lecture date: July 24, 2025
  tl;dr: An introduction to the course content, logistics, policies and background.
  [ slides ] [ recording ]
  

  Suggested Readings:

  • A Survey of Deep Learning: From Activations to Transformers
• 2. Introduction to Language Models
  Lecture date: August 04, 2025
  tl;dr: Introduction to language modelling, RNNs, backpropagation through time, LSTMs, GRUs.
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Chapter-3, Speech and Language Processing
  • Backpropagation Through Time: What It Does and How to Do It
  • The Unreasonable Effectiveness of Recurrent Neural Networks
  • Learning long-term dependencies with gradient descent is difficult
  • On the difficulty of training Recurrent Neural Networks
  • Understanding LSTM Networks
• 3. Sequence to Sequence Model, Attention Mechanism
  Lecture date: August 06, 2025
  tl;dr: Training of seq2seq models, seq2seq attention, self attention
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Sequence to Sequence Learning with Neural Networks
  • Neural Machine Translation by Jointly Learning to Align and Translate
  • Attention and Augmented Recurrent Neural Networks
• 4. Transformer: Architecture and Training
  Lecture date: August 07, 2025
  tl;dr: Transformer Architecture, Pre-training and Fine-tuning of BERT
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Attention Is All You Need
  • The Illustrated Transformer
  • Transformer Explainer
  • Layer Normalization
  • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• 5. Pre-training and Instruction Tuning
  Lecture date: August 11, 2025
  tl;dr: Pre-training strategies of Encoder-only, Encoder-decoder and Decoder-only models; Instruction Tuning, Weighted Instruction Tuning
  [ slides ] [ recordingA ] [ recordingB ]
  

  Suggested Readings:

  • Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  • BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
  • SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions
  • Orca: Progressive Learning from Complex Explanation Traces of GPT-4
  • On the Effect of Instruction Tuning Loss on Generalization
  • Instruction Tuning for Large Language Models: A Survey
• 6. Tokenization
  Lecture date: August 13, 2025
  tl;dr: BPE, WordPiece, Unigram Tokenization
  [ slides ] [ recording ]
  

  Suggested Readings:

  • (BPE) Neural Machine Translation of Rare Words with Subword Units
  • (WordPiece Tokenization) Japanese and Korean Voice Search
  • (Unigram Tokenization) Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
• 7. RLHF: Part 01
  Lecture date: August 18, 2025
  tl;dr: Basiscs of RLHF, Reward models, BT and Non-BT models
  [ slides ] [ recording ]
  
• 8. RLHF: Part 02
  Lecture date: August 20, 2025
  tl;dr: Reward Maximization: Best-of-N policy, Objective Formulation, Vanilla Policy Gradient, Actor/Critic Methods
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
• 9. RLHF: Part 03
  Lecture date: August 21, 2025
  tl;dr: GRPO, PPO, TRPO
  [ slides ] [ recording ]
  

  Suggested Readings:

  • OpenAI Spinning Up (documentation and introductory blogs on Deep RL)
  • Trust Region Policy Optimization
  • High-Dimensional Continuous Control Using Generalized Advantage Estimation
  • Proximal Policy Optimization Algorithms
  • DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (for GRPO)
  • Training language models to follow instructions with human feedback
• 10. RLHF: Part 04
  Lecture date: August 25, 2025
  tl;dr: PPO/GRPO for LLM alignment, DPO, PPO vs DPO
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • Direct Language Model Alignment from Online AI Feedback
• 11. Efficient LLMs: Part 01
  Lecture date: August 27, 2025
  tl;dr: Efficient distributed training - memory profiling, gradient checkpointing, gradient accumulation
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Transformer Math (Part 1) - Counting Model Parameters
  • Blog on Gradient Checkpointing
  • Blog on Gradient Accumulation
• 12. Efficient LLMs: Part 02
  Lecture date: August 28, 2025
  tl;dr: Efficient distributed training - data parallelism, ZeRO (1,2,3), FSDP
  [ slides ] [ recordings ]
  

  Suggested Readings:

  • Blog: Data-Parallel Distributed Training of Deep Learning Models
  • ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
  • ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters
  • Blog on FSDP
  • PyTorch FSDP Tutorial
• 13. Efficient LLMs: Part 03
  Lecture date: September 1, 2025
  tl;dr: Efficient distributed training - tensor parallelism, sequence parallelism, context parallelism, ring attention
  [ slides ] [ recording ]
  

  Suggested Readings:

  • Reducing Activation Recomputation in Large Transformer Models (Tensor Parallelism, Sequence Parallelism, Pipeline Parallelism)
  • Ring Attention with Blockwise Transformers for Near-Infinite Context
  • PyTorch Blog: Large Scale Transformer model training with Tensor Parallel (TP)
• 14. Efficient LLMs: Part 04
  Lecture date: September 3, 2025
  tl;dr: Pipeline Parallelism (AFAB, 1F1B), basics of GPU, Flash attention
  [ slides ] [ video ]
  

  Suggested Readings:

  • PipeDream: Fast and Efficient Pipeline Parallel DNN Training
  • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  • Flash attention blog
• 15. Efficient LLMs: Part 05
  Lecture date: September 4, 2025
  tl;dr: Training vs. Inference: Forward Pass, Inference, KV Cache usage and Management(vLLMs, KV Blocks, Paged Attention)
  [ slides ] [ video ]
  

  Suggested Readings:

  • Decoder-Only Transformers: The Workhorse of Generative LLMs
  • The KV Cache: Memory Usage in Transformers
  • Fast LLM Serving with vLLM and PagedAttention
• 16. Efficient LLMs: Part 06
  Lecture date: September 8, 2025
  tl;dr: Training vs. Inference(in code), Mixture of Experts Architecture, Efficient LLMs Recap
  [ slides ] [ video ]
  

  Suggested Readings:

  • Towards Understanding Mixture of Experts in Deep Learnings
  • A Survey on Mixture of Experts in Large Language Models
  • Blog on Mixture of Experts
• 17. Parameter-Efficient Fine-Tuning (PEFT)
  Lecture date: September 10, 2025
  tl;dr: Additive, Selective and Re-parameterization PEFT Techniques
  [ slides ] [ video ]
  

  Suggested Readings:

  • Parameter-Efficient Transfer Learning for NLP
  • Prefix-Tuning: Optimizing Continuous Prompts for Generation
  • The Power of Scale for Parameter-Efficient Prompt Tuning
  • BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  • Training Neural Networks with Fixed Sparse Masks
  • Parameter-Efficient Fine-Tuning without Introducing New Latency
  • LoRA: Low-Rank Adaptation of Large Language Models
  • AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
  • DoRA: Weight-Decomposed Low-Rank Adaptation
  • QLoRA: Efficient Finetuning of Quantized LLMs
  • Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
  • Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
• 18. Model Compression
  Lecture date: September 11, 2025
  tl;dr: Different types of model pruning techniques
  [ slides ] [ video ]
  

  Suggested Readings:

  • Learning both Weights and Connections for Efficient Neural Networks
  • SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
  • SliceGPT: Compress Large Language Models by Deleting Rows and Columns
  • You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning
• 19. Knowledge Distillation
  Lecture date: September 18, 2025
  tl;dr: Different techniques for knowledge distillation in LLMs
  [ slides ] [ video ]
  

  Suggested Readings:

  • Distilling the Knowledge in a Neural Network
  • Sequence-Level Knowledge Distillation
  • MiniLLM: Knowledge Distillation of Large Language Models
  • On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
  • BERT Learns to Teach: Knowledge Distillation with Meta Learning
  • A Good Learner can Teach Better: Teacher-Student Collaborative Knowledge Distillation
  • On the Generalization vs Fidelity Paradox in Knowledge Distillation
• 20. Retrieval-based LMs: Part 01
  Lecture date: September 22, 2025
  tl;dr: Motivation behind retrieval-augmented LMs, Retriever pipeline, different retrieval methods (sparse and dense)
  [ slides ] [ video ]
  

  Suggested Readings:

  • Scoring, term weighting and the vector space model (chapter-6)
  • Dense Passage Retrieval for Open-Domain Question Answering
• 21. Retrieval-based LMs: Part 02
  Lecture date: September 24, 2025
  tl;dr: Cross-Encoder Reranking, Token level Dense Retrieval (COLBERT), Graph RAG, Hippo RAG
  [ slides ] [ video ]
  

  Suggested Readings:

  • ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  • Precise Zero-Shot Dense Retrieval without Relevance Labels
  • How to Build a RAG System on a Knowledge Graph
  • HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
• 22. LLMs and Tools: Incorporating Tools during Fine-tuning
  Lecture date: September 25, 2025
  tl;dr: Tool augmentation in LLMs during fine-tuning, TALM, PAL, Toolformer, WebGPT
  [ slides ] [ video ]
  

  Suggested Readings:

  • Training Verifiers to Solve Math Word Problems
  • TALM: Tool Augmented Language Models
  • PAL: Program-aided Language Models
  • Toolformer: Language Models Can Teach Themselves to Use Tools
  • WebGPT: Browser-assisted question-answering with human feedback
• 23. LLMs and Tools: Teaching LLMs to Use External APIs
  Lecture date: October 6, 2025
  tl;dr: Teaching LLMs to use external APIs - APIBench, ToolAlpaca, ToolBench, APIGen, Granite-Function Calling Model
  [ slides ] [ video ]
  

  Suggested Readings:

  • Gorilla: Large Language Model Connected with Massive APIs
  • ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
  • APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
  • Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks
• 24. LLMs and Tools: Automating Complex Tasks
  Lecture date: October 8, 2025
  tl;dr: Teaching LLMs to use external APIs - ToolFlow, MAGNET
  [ slides ] [ video ]
  

  Suggested Readings:

  • ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
  • Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translation
• 25. LLM Reasoning: Part 01
  Lecture date: October 9, 2025
  tl;dr: How to make LLMs Reason - CoT Prompting, Thought Process Optimization, RL
  [ slides ] [ video ]
  

  Suggested Readings:

  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Thinking LLMS: General Instruction Following with Thought Generation
  • Cognitive Behaviors that Enable Self-Improving Reasoners
• 26. LLM Reasoning: Part 02
  Lecture date: October 13, 2025
  tl;dr: Deepseek-R1-zero, Reasoning in weaker models-Teaching Restarts, Backtracking (MCTS)
  [ slides ] [ video ]
  

  Suggested Readings:

  • ASTRO: Teaching Language Models to Reason by Reflecting and Backtracking In-Context
  • DeepSeek-R1: Incentivizing Reasoning via RL
  • Understanding Reasoning LLMs
• 27. LLM Reasoning: Part 03 (Test-Time Scaling)
  Lecture date: October 15, 2025
  tl;dr: Scaling test-time compute with reasoning models - parallel scaling, sequential scaling, hybrid scaling, internal scaling
  [ slides ] [ video ]
  

  Suggested Readings:

  • A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
  • Self-Refine: Iterative Refinement with Self-Feedback
  • s1: Simple test-time scaling
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • OpenAI o1 System Card
  • Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
  • First Finish Search: Efficient Test-Time Scaling in Large Language Models
  • Position: Enough of Scaling LLMs! Lets Focus on Downscaling
• 28. LLMs and Tools: Agentic Workflow: Part 01
  Lecture date: October 16, 2025
  tl;dr: Automating Complex Tasks - ReACT, Self-Refine, Reflexion
  [ slides ] [ video ]
  

  Suggested Readings:

  • ReAct: Synergizing Reasoning and Acting in Language Models
  • Self-Refine: Iterative Refinement with Self-Feedback
  • Reflexion: Language Agents with Verbal Reinforcement Learning
• 29. LLMs and Tools: Agentic Workflow: Part 02
  Lecture date: October 22, 2025
  tl;dr: Memory management in AI agents, enabling a small-sized LLM to approach larger proprietary model performance
  [ slides ]
  

  Suggested Readings:

  • MemGPT: Towards LLMs as Operating Systems
  • SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
• 30. LLMs and Tools: Agentic Workflow: Part 03
  Lecture date: October 23, 2025
  tl;dr: Agentic Protocols - Model Context Protocol (MCP), Agent2Agent (A2A) Protocol, agents.json
  [ slides ]
  

  Suggested Readings:

  • A Survey of AI Agent Protocols
  • Blog: What Is the Model Context Protocol (MCP) and How It Works
  • Blog: What is A2A?
• 31. Alternative Models: RWKV
  Lecture date: October 25, 2025
  tl;dr: Receptance Weighted Key Value (RWKV)
  [ slides ]
  

  Suggested Readings:

  • RWKV: Reinventing RNNs for the Transformer Era
• 32. Alternative Models: SSMs
  Lecture date: October 27, 2025
  tl;dr: State Space Machines (SSMs)
  [ slides ]
  

  Suggested Readings:

  • Blog: Introduction to State Space Models (SSM)
• 33. Multimodal Encoder Models
  Lecture date: October 29, 2025
  tl;dr: Multimodal encoder models - ViT, VisualBERT, VilBERT, CLIP, LayoutLMv2, ViT with registers, DinoV3, VideoCLIP, ImageBind
  [ slides ]
  

  Suggested Readings:

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • VisualBERT: A Simple and Performant Baseline for Vision and Language
  • ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
  • Learning Transferable Visual Models From Natural Language Supervision
  • LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
  • Vision Transformers Need Registers
  • DINOv3
  • VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
  • IMAGEBIND: One Embedding Space To Bind Them All
• 34. Text Generation with Multimodal Inputs
  Lecture date: October 30, 2025
  tl;dr: Multimodal text generation architectures - Frozen, Flamingo, BLIP, BLIP2, mPLUG, LLaVa, Video-LLaMA, MiniGPT-4, MiniCPM-V, UI-TARS
  [ slides ]
  

  Suggested Readings:

  • Visual Instruction Tuning
  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
  • BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • MiniCPM-V: A GPT-4V Level MLLM on Your Phone
  • Flamingo: a Visual Language Model for Few-Shot Learning
  • UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• 35. Alternative Models: Discretized SSMs & Structured Discrete SSMs
  Lecture date: November 3, 2025
  tl;dr: Discretized State Space Machines, S4 and Mamba
  [ slides-1 ] [ slides-2 ]
  

  Suggested Readings:

  • Efficiently Modeling Long Sequences with Structured State Spaces
  • Mamba: Linear-Time Sequence Modeling with Selective State Spaces
  • Blog: The Annotated S4
• 36. Physics of LLMs: Knowledge Storage and Extraction
  Lecture date: November 6, 2025
  tl;dr: Understanding why augmented pre-training data is essential for LLM knowledge extraction
  [ slides ]
  

  Suggested Readings:

  • Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
• 37. Physics of LLMs: Knowledge Manipulation, Knowledge Capacity Scaling Laws and Reasoning
  Lecture date: November 10, 2025
  tl;dr: Understanding LLM reasoning, knowledge manipulation, and capacity scaling laws
  [ slides ]
  

  Suggested Readings:

  • Physics of Language Models: Part 3.2, Knowledge Manipulation
  • Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
  • Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
  • Physics of Language Models: Part 1, Learning Hierarchical Language Structures
• 38. Interpretability of LLMs
  Lecture date: November 12, 2025
  tl;dr: Local and Global Explanation-based Analysis, Sparse Autoencoders, Activation patching and steering
  [ slides ]
  

  Suggested Readings:

  • Sparse Autoencoders Find Highly Interpretable Features In Language Models
  • Finding neurons in a haystack: Case studies with sparse probing
  • Sparse Crosscoders for Cross-Layer Features and Model Diffing
  • Transcoders Find Interpretable LLM Feature Circuits