Lectures

You can download the lectures here. 🔔 Subscribe to our newsletter for the latest updates on LLMs!

• 

  1. Course Introduction
  Lecture date: July 22, 2024 (Monday)
  tl;dr: An introduction to the course content, logistics, policies and background.
  [slides] [video]
  

  Suggested Readings:

  • A Survey of Deep Learning: From Activations to Transformers
• 

  2. Introduction to Natural Language Processing
  Lecture date: July 24, 2024 (Wednesday)
  tl;dr: An introduction to the terminologies and problems in NLP.
  [slides] [video]
  

  Suggested Readings:

  • The Evolution of NLP: Past, Present, and Future
• 

  3.1. Introduction to Language Models: Statistical Language Modeling
  Lecture date: July 25, 2024 (Thursday)
  tl;dr: An introduction to statistical language modeling.
  [slides] [video]
  

  Suggested Readings:

  • Chapter-3, Speech and Language Processing
• 

  3.2. Introduction to Language Models: Advanced Smoothing and Evaluation
  Lecture date: July 29, 2024 (Monday)
  tl;dr: Discussion on the advanced smoothing techniques, and the intrinsic and extrinsic methods for evaluation of language models.
  [slides] [video]
  

  Suggested Readings:

  • Chapter-3, Speech and Language Processing
• 

  4.1. Word Representation: Word2vec
  Lecture date: July 31, 2024 (Wednesday)
  tl;dr: Various methods for representing words as vectors – count-based methods, prediction-based methods (Word2vec, fastText).
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Chapter-6, Speech and Language Processing
  • Efficient Estimation of Word Representations in Vector Space
  • Enriching Word Vectors with Subword Information
• 

  4.2. Word Representation: GloVe
  Lecture date: August 1, 2024 (Thursday)
  tl;dr: Combining count-based and prediction-based methods for learning word representations – GloVe.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • GloVe: Global Vectors for Word Representation

  Optional Readings on Bias Captured by Word Embeddings:

  • Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
  • Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
  • Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• 

  5.1. Neural Language Models: RNNs
  Lecture date: August 1, 2024 (Thursday)
  tl;dr: Language modeling using different variants of neural networks (CNN, RNN). A quick overview of the training algorithm of RNNs – Backpropagation through time.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Backpropagation Through Time: What It Does and How to Do It
  • The Unreasonable Effectiveness of Recurrent Neural Networks
  • Chapter-10 (Sequence Modeling: Recurrent and Recursive Nets), ‘Deep Learning’ book by I. Goodfellow, Y. Bengio and A. Courville
• 

  5.2. Neural Language Models: LSTMs and GRUs
  Lecture date: August 2, 2024 (Friday)
  tl;dr: Problems with RNNs – vanishing gradients. Architectural modifications for solving vanishing gradient problem – LSTMs and GRUs.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Learning long-term dependencies with gradient descent is difficult
  • On the difficulty of training Recurrent Neural Networks
  • Understanding LSTM Networks
• 

  5.3. Neural Language Models: Sequence-to-Sequence and Attention
  Lecture date: August 2, 2024 (Friday)
  tl;dr: Neural network architectures for modeling sequence-to-sequence problems. Introduction to attention mechanism in RNNs.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Sequence to Sequence Learning with Neural Networks
  • Neural Machine Translation by Jointly Learning to Align and Translate
  • Attention and Augmented Recurrent Neural Networks
• 

  6.1. Introduction to Transformer: Self-Attention and Multi-Head Attention
  Lecture date: August 5, 2024 (Monday)
  tl;dr: Getting rid of recurrent connections – self-attention, multi-head attention, masked decoding. Introduction to positional encoding.
  [slides] [video]
  

  Suggested Readings:

  • Attention Is All You Need
  • The Illustrated Transformer
  • Transformer Explainer
• 

  6.2. Introduction to Transformer: Positional Encoding and Layer Normalization
  Lecture date: August 7, 2024 (Wednesday)
  tl;dr: Discussion on various types on positional encoding methods (Absolute Positional Encoding, Relative Positional Encoding, Rotary Positional Encoding). Understanding Layer Normalization.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • RoFormer: Enhanced Transformer with Rotary Position Embedding
  • Layer Normalization
  • Build Better Deep Learning Models with Batch and Layer Normalization
• 

  7. Pre-training Strategies
  Lecture date: August 8, 2024 (Thursday)
  tl;dr: Discussion on ELMo. Understanding the pre-training strategies of encoder-only transformer (BERT) – Masked Language Modeling.
  [slides] [video]
  

  Suggested Readings:

  • Deep contextualized word representations
  • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• 

  8.1. Advanced Attention Mechanisms-I
  Lecture date: August 21, 2024 (Wednesday)
  tl;dr: Understanding KV Caching. Discussion on the efficient attention mechanisms – multi-query attention, grouped query attention, sliding window attention.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Longformer: The Long-Document Transformer
  • Fast Transformer Decoding: One Write-Head is All You Need
  • GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
  • Transformers KV Caching Explained
• 

  8.2. Advanced Attention Mechanisms-II
  Lecture date: August 22, 2024 (Thursday)
  tl;dr: Understanding Flash Attention.
  [slides] [video]
  

  Suggested Readings:

  • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• 

  9. Tokenization Strategies
  Lecture date: August 26, 2024 (Monday)
  tl;dr: Understanding different techniques for tokenization – Byte-Pair Encoding (BPE), WordPiece, Unigram tokenization.
  [slides] [video]
  

  Suggested Readings:

  • (BPE) Neural Machine Translation of Rare Words with Subword Units
  • (WordPiece Tokenization) Japanese and Korean Voice Search
  • (Unigram Tokenization) Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
• 

  10.1. Mixture of Experts-I
  Lecture date: August 28, 2024 (Wednesday)
  tl;dr: Discussion on the Mixture of Experts (MoE) architectural paradigm – Sparse MoE, Routing Mechanisms and Learning Dynamics.
  [slides] [video]
  

  Suggested Readings:

  • Adaptive Mixtures of Local Experts
  • Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
  • Towards Understanding the Mixture-of-Experts Layer in Deep Learning
• 

  10.2. Mixture of Experts-II
  Lecture date: August 29, 2024 (Thursday)
  tl;dr: Understanding the architecture of Switch Transformers, Mixtral. Discussion on Model Parallelism.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
  • Mixtral of Experts
  • Paradigms of Parallelism
  • No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
• 

  11. Scaling Laws
  Lecture date: August 31, 2024 (Saturday)
  tl;dr: Discussion on emergent abilities of LLMs. Understanding the empirical scaling laws for neural language model performance on the cross-entropy loss – Kaplan laws, Chinchilla scaling laws. Discussion on an alternative perspective on emergent abilities – are these abilities really 'emergent'?
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Scaling Laws for Neural Language Models
  • Emergent Abilities of Large Language Models
  • Training Compute-Optimal Large Language Models
  • Are Emergent Abilities of Large Language Models a Mirage?
• 

  12.1. Pre-training of Causal LMs and In-context Learning
  Lecture date: September 2, 2024 (Monday)
  tl;dr: Looking into the procedure for pre-training of causal/auto-regressive language models. Discussion on the in-context learning ability of LLMs.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Improving Language Understanding by Generative Pre-Training
  • Language Models are Unsupervised Multitask Learners
  • Language Models are Few-Shot Learners
  • Learning To Retrieve Prompts for In-Context Learning
  • Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
  • Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
• 

  12.2. Instruction Tuning
  Lecture date: September 4, 2024 (Wednesday)
  tl;dr: Discussion on instruction tuning of LLMs – data collection, loss function, and properties of tuned models.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Instruction Tuning for Large Language Models: A Survey
  • The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
  • SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions
  • WizardLM: Empowering Large Language Models to Follow Complex Instructions
  • Orca: Progressive Learning from Complex Explanation Traces of GPT-4
• 

  13.1. Alignment of Language Models: Reward Maximization-I
  Lecture date: September 5, 2024 (Thursday)
  tl;dr: Looking into the reward model for alignment – modeling the alignment procedure as reinforcement learning, the architecture of reward model, training the reward model, gathering preference data (RLHF vs RLAIF), reward maximization objective.
  [slides] [scribe] [video]
  
• 

  13.2. Alignment of Language Models: Reward Maximization-II
  Lecture date: September 7, 2024 (Saturday)
  tl;dr: Looking into different algorithms for training the policy model, which is the LLM, to maximize the reward – REINFORCE, PPO.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Proximal Policy Optimization Algorithms
  • Training language models to follow instructions with human feedback
  • Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
• 

  13.3. Alignment of Language Models: Contrastive Learning
  Lecture date: September 9, 2024 (Monday)
  tl;dr: Discussion on the Direct Preference Optimization (DPO) algorithm.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • Direct Language Model Alignment from Online AI Feedback
• 

  14.1. Parameter Efficient Fine-Tuning (PEFT)
  Lecture date: September 19, 2024 (Thursday)
  tl;dr: Discussion on various PEFT techniques – prompt tuning, prefix tuning, adapters, low-rank adaptation (LoRA).
  [slides] [video]
  

  Suggested Readings:

  • The Power of Scale for Parameter-Efficient Prompt Tuning
  • Prefix-Tuning: Optimizing Continuous Prompts for Generation
  • Parameter-Efficient Transfer Learning for NLP
  • LoRA: Low-Rank Adaptation of Large Language Models
• 

  14.2. Quantization, Pruning & Distillation
  Lecture date: September 23, 2024 (Monday)
  tl;dr: Discussion on various model compression techniques – post-training quantization, QLoRA, magnitude and structured pruning, knowledge distillation.
  [slides] [video]
  

  Suggested Readings:

  • LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
  • QLoRA: Efficient Finetuning of Quantized LLMs
  • Structured Pruning Learns Compact and Accurate Models
  • A Simple and Effective Pruning Approach for Large Language Models
  • Distilling the Knowledge in a Neural Network
  • Sequence-Level Knowledge Distillation
• 

  15.1. Efficient LLM Decoding-I
  Lecture date: September 25, 2024 (Wednesday)
  tl;dr: Discussion on various efficient inference/decoding techniques – KV caching, paged attention and vLLM.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • LLM Inference Serving: Survey of Recent Advances and Opportunities
  • Efficient Memory Management for Large Language Model Serving with PagedAttention
• 

  15.2. Efficient LLM Decoding-II
  Lecture date: September 26, 2024 (Thursday)
  tl;dr: Discussion on various efficient decoding techniques – flash decoding, speculative decoding, Medusa and tree attention, prompt-lookup decoding, lookahead decoding.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Flash-Decoding for long-context inference
  • Fast Inference from Transformers via Speculative Decoding
  • Accelerating Large Language Model Decoding with Speculative Sampling
  • MEDUSA: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
  • Break the Sequential Dependency of LLM Inference Using LOOKAHEAD DECODING
• 

  16.1. Retrieval-based Language Models-I
  Lecture date: September 30, 2024 (Monday)
  tl;dr: Discussion on the motivation behind retrieval-based LMs and various retrieval methods – sparse and dense retrieval, cross-encoder reranking, differentiable search index, table-of-contents aware search.
  [slides] [video]
  

  Suggested Readings:

  • Chapter-6, Introduction to Information Retrieval
  • Reading Wikipedia to Answer Open-Domain Questions
  • Dense Passage Retrieval for Open-Domain Question Answering
  • Unsupervised Dense Information Retrieval with Contrastive Learning
  • Passage Re-ranking with BERT
  • ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  • Precise Zero-Shot Dense Retrieval without Relevance Labels
  • Transformer Memory as a Differentiable Search Index
• 

  16.2. Retrieval-based Language Models-II
  Lecture date: October 3, 2024 (Thursday)
  tl;dr: Looking into various retrieval-based LMs – kNN LM, RETRO, REALM, RAG. Discussion on different training methods for retrieval-augmented LMs and their limitations.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Generalization through Memorization: Nearest Neighbor Language Models
  • Improving language models by retrieving from trillions of tokens
  • REALM: Retrieval-Augmented Language Model Pre-Training
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • In-Context Retrieval-Augmented Language Models
  • REPLUG: Retrieval-Augmented Black-Box Language Models
• 

  17.1. Multimodal Models-I
  Lecture date: October 7, 2024 (Monday)
  tl;dr: Understanding the architecture and pre-training strategies of multimodal models – the focus of this lecture is on multimodal understanding involving two modalities (image and text).
  [slides] [video]
  

  Suggested Readings:

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • VisualBERT: A Simple and Performant Baseline for Vision and Language
  • ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
  • Learning Transferable Visual Models From Natural Language Supervision
  • LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
  • VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
  • IMAGEBIND: One Embedding Space To Bind Them All
• 

  17.2. Multimodal Models-II
  Lecture date: October 14, 2024 (Monday)
  tl;dr: Discussion on text generation with multimodal inputs.
  [slides] [video]
  

  Suggested Readings:

  • Multimodal Few-Shot Learning with Frozen Language Models
  • Flamingo: a Visual Language Model for Few-Shot Learning
  • BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connection
  • Visual Instruction Tuning
  • Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
  • MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
• 

  18.1. LLMs and Tools: Tool Augmentation
  Lecture date: October 16, 2024 (Wednesday)
  tl;dr: Understanding how we can enable pre-trained language models to use external tools and incorporate tool usage during fine-tuning.
  [slides] [video]
  

  Suggested Readings:

  • Training Verifiers to Solve Math Word Problems
  • TALM: Tool Augmented Language Models
  • PAL: Program-aided Language Models
  • Toolformer: Language Models Can Teach Themselves to Use Tools
  • WebGPT: Browser-assisted question-answering with human feedback
• 

  18.2. LLMs and Tools: Function Calling
  Lecture date: October 17, 2024 (Thursday)
  tl;dr: Discussion on how we can teach LLMs to use APIs and call appropriate functions when required.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • Gorilla: Large Language Model Connected with Massive APIs
  • ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
  • APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
  • Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks
• 

  18.3. LLMs and Tools: Agentic Workflow
  Lecture date: October 19, 2024 (Saturday)
  tl;dr: Discussion on how we can automate complex, multi-step tasks – developing LLM-based agents.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • ReAct: Synergizing Reasoning and Acting in Language Models
  • Self-Refine: Iterative Refinement with Self-Feedback
  • Reflexion: Language Agents with Verbal Reinforcement Learning
  • ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
  • HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
• 

  19. Reasoning in LLMs
  Lecture date: October 21, 2024 (Monday)
  tl;dr: Looking into different types of reasoning tasks and various techniques (like, Chain-of-Thought prompting, backward chaining, etc.) facilitating LLMs to solve these tasks. Overview of various reasoning benchmarks and discussion on whether LLMs can truly reason and plan, highlighting both current capabilities and limitations.
  [slides] [video]
  

  Suggested Readings:

  • Natural Language Reasoning, A Survey
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning
  • The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A”
  • Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
  • Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
  • On the Planning Abilities of Large Language Models: A Critical Investigation
  • Can Large Language Models Reason and Plan?
  • LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks
• 

  20. Long Context LLMs: Challenges & Solutions
  Lecture date: October 28, 2024 (Monday)
  tl;dr: Discussion on the challenges LLMs face while processing long input contexts and various techniques to enable LLMs to handle long contexts effectively.
  [slides] [video]
  

  Suggested Readings:

  • LongNet: Scaling Transformers to 1,000,000,000 Tokens
  • Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
  • Extending Context Window of Large Language Models via Positional Interpolation
  • Lost in the Middle: How Language Models Use Long Contexts
• 

  21. Knowledge Editing
  Lecture date: October 30, 2024 (Wednesday)
  tl;dr: Discussion on some of the techniques for editing 'knowledge' stored in LLMs without further re-training.
  [slides] [video01] [video02] [video03] [video04]
  

  Suggested Readings:

  • Editing Factual Knowledge in Language Models
  • Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
  • Locating and Editing Factual Associations in GPT
  • Knowledge Editing for Large Language Models: A Survey
• 

  22. Self-evolving LLMs
  Lecture date: November 7, 2024 (Thursday)
  tl;dr: Discussion on self-evolution approaches that enable LLM to autonomously acquire, refine, and learn from experiences generated by the model itself.
  [slides] [scribe] [video]
  

  Suggested Readings:

  • A Survey on Self-Evolution of Large Language Models
  • STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning
  • Self-Instruct: Aligning Language Models with Self-Generated Instructions
  • Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
  • Self-Refine: Iterative Refinement with Self-Feedback
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
• 

  23. An Alternate Formulation of Transformers
  Lecture date: November 11, 2024 (Monday)
  tl;dr: Understanding the residual stream perspective of Transformers and decomposing the role of each model component.
  [scribe] [video]
  

  Suggested Readings:

  • A Mathematical Framework for Transformer Circuits
  • A Primer on the Inner Workings of Transformer-based Language Models
• 

  24. Interpretability: Demystifying the Black-Box LMs
  Lecture date: November 13, 2024 (Wednesday)
  tl;dr: Discussion on various interpretability techniques to decipher the inner workings of LLMs.
  [slides] [scribe]
  

  Suggested Readings:

  • Probing Classifiers: Promises, Shortcomings, and Advances
  • Mechanistic?
  • Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small
  • Towards Automated Circuit Discovery for Mechanistic Interpretability
  • Towards Best Practices of Activation Patching in Language Models: Metrics and Methods
  • Attribution Patching: Activation Patching At Industrial Scale
  • In-context Learning and Induction Heads
  • How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning
  • Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
  • Towards Monosemanticity: Decomposing Language Models With Dictionary Learning