Lectures

🔔 Subscribe to our newsletter for the latest updates on LLMs!

• 1. Course Introduction
  Lecture date: July 23, 2026
  tl;dr: An introduction to the course content, logistics, policies and background.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • A Survey of Deep Learning: From Activations to Transformers
• 2. Introduction to Language Models and Neural Language Models
  Lecture date: July 27, 2026
  tl;dr: Language modelling, Word2Vec, Fixed-window LM, RNN, LSTM, GRU, Sampling.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Chapter-3, Speech and Language Processing
  • Sequence to Sequence Learning with Neural Networks
  • Neural Machine Translation by Jointly Learning to Align and Translate
  • Attention and Augmented Recurrent Neural Networks
  • The Unreasonable Effectiveness of Recurrent Neural Networks
  • Learning long-term dependencies with gradient descent is difficult
  • On the difficulty of training Recurrent Neural Networks
  • Understanding LSTM Networks
• 3. Introduction to Transformers - I
  Lecture date: July 30, 2026
  tl;dr: Self-Attention, Masked Attention, Transformer architecture and internals.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Attention Is All You Need
  • The Illustrated Transformer
  • Transformer Explainer
  • Layer Normalization
• 4. Introduction to Transformers - II
  Lecture date: August 03, 2026
  tl;dr: Pre-training strategies, Instruction Fine-tuning, and Parameter Efficient Fine-tuning methods.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  • Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  • BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
  • SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions
  • Orca: Progressive Learning from Complex Explanation Traces of GPT-4
  • On the Effect of Instruction Tuning Loss on Generalization
  • Instruction Tuning for Large Language Models: A Survey
  • Parameter-Efficient Transfer Learning for NLP
  • Prefix-Tuning: Optimizing Continuous Prompts for Generation
  • The Power of Scale for Parameter-Efficient Prompt Tuning
  • BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  • Training Neural Networks with Fixed Sparse Masks
  • Parameter-Efficient Fine-Tuning without Introducing New Latency
  • LoRA: Low-Rank Adaptation of Large Language Models
  • AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
  • DoRA: Weight-Decomposed Low-Rank Adaptation
  • QLoRA: Efficient Finetuning of Quantized LLMs
  • Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
  • Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
• 5. Introduction to Transformers - III
  Lecture date: August 06, 2026
  tl;dr: Alignment, Reinforcement Learning from Human Feedback, DPO.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
  • Variance Reduction based Partial Trajectory Reuse to Accelerate Policy Gradient Optimization
  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• 6. Privacy - I: Introduction to Privacy and Text Sanitization
  Lecture date: August 10, 2026
  tl;dr: Introduction to privacy, regulatory frameworks, real-world failures, PII-based text anonymization, text sanitization.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • LLM Privacy: From Principles to Practice
  • Do foundation model providers comply with the draft EU AI Act?
  • Robust De-anonymization of Large Sparse Datasets
  • Extracting Training Data from Large Language Models
  • Data Privacy Handbook
  • Text Sanitization Beyond Specific Domains: Zero-Shot Redaction & Substitution with Large Language Models
  • Reducing Privacy Risks in Online Self-Disclosures with Language Models
  • Beyond Memorization: Violating Privacy Via Inference with Large Language Models
  • Large Language Models are Advanced Anonymizers
  • Robust Utility-Preserving Text Anonymization Based on Large Language Models
• 7. Privacy - II: Privacy Attacks in LLMs
  Lecture date: August 13, 2026
  tl;dr: Memorization, Training data extraction attack, Membership Inference Attack, Attack Transfer.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
  • Membership Inference Attacks From First Principles
  • Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
  • Do Membership Inference Attacks Work on Large Language Models?
  • Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysis
  • Optimization and Robustness-Informed Membership Inference Attacks for LLMs
• 8. Privacy - III: Privacy Attacks and Defenses in LLMs
  Lecture date: August 17, 2026
  tl;dr: Prompt and ICL leakage, Prompt injection, Introduction to Differential Privacy.
  [ slides 1 ] [ scribe 1 ] [ slides 2 ] [ scribe 2 ]
  

  Suggested Readings:

  • Effective Prompt Extraction from Language Models
  • PLeak: Prompt Leaking Attacks against Large Language Model Applications
  • Prompt Stealing Attacks Against Large Language Models
  • Has My System Prompt Been Used? Large Language Model Prompt Membership Inference
  • ContextLeak: Auditing Leakage in Private In-Context Learning Methods
  • Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
  • The ABCs of Differential Privacy
• 9. Privacy - IV: Privacy Defenses in LLMs
  Lecture date: August 20, 2026
  tl;dr: Differential privacy, DP-SGD, DP-based Sanitization.
  [ slides ] [ scribe ] [ notepad ]
  

  Suggested Readings:

  • Deep Learning with Differential Privacy
  • Pre-training Differentially Private Models with Limited Public Data
  • VaultGemma: A Differentially Private Gemma Model
  • Large Language Models Can Be Strong Differentially Private Learners
  • Differentially Private Fine-tuning of Language Models
  • DYNTEXT:Semantic-Aware Dynamic Text Sanitization for Privacy-Preserving LLM Inference
• 10. Security and Robustness - I: Finding and Evaluating Adversarial Inputs
  Lecture date: August 24, 2026
  tl;dr: Continuous adversarial examples, discrete token optimization, black-box jailbreak search, long-context attacks and robustness evaluation.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Towards Deep Learning Models Resistant to Adversarial Attacks
  • HotFlip: White-Box Adversarial Examples for Text Classification
  • Universal and Transferable Adversarial Attacks on Aligned Language Models
  • Detecting Language Model Attacks with Perplexity
  • AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
• 11. Security and Robustness - II: Why Safety Training Fails
  Lecture date: August 27, 2026
  tl;dr: Adversarial transfer, refusal mechanisms and shallow alignment, fine-tuning attacks, training-data poisoning, persistent backdoors.
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Jailbreaking Black Box Large Language Models in Twenty Queries
  • Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
  • Many-shot Jailbreaking
  • HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
  • A StrongREJECT for Empty Jailbreaks
• 12. Security and Robustness - III: Monitoring and Securing LLM Agents
  Lecture date: August 31, 2026
  tl;dr: Agent Foundations, Indirect Prompt Injection, Memory Poisoning, Authorization and Architectural Enforcement
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
  • AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
  • Memory Injection Attacks on LLM Agents via Query-Only Interaction
  • Defeating Prompt Injections by Design
• 13. Bias and Fairness - I: Introduction
  Lecture date: September 7, 2026
  tl;dr: Foundations, Social and Algorithmic Bias
  [ slides ] [ scribe ]
  

  Suggested Readings:

  • Semantics derived automatically from language corpora contain human-like biases
  • Review into bias in algorithmic decision-making
  • Monkeys reject unequal pay
  • Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
  • Equality of Opportunity in Supervised Learning
  • On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
  • How Far Can Bias Go? Tracing Bias from Pre-Training Data to Alignment
  • Training language models to follow instructions with human feedback
• 14. Bias and Fairness - II: Metrics
  Lecture date: September 10, 2026
  tl;dr: Measuring Bias, Fairness Criteria and Benchmarks, Bias in Pretraining, Tokenization and RLHF
  [ slides ]
  

  Suggested Readings:

  • AI Insights: Large language models (LLMs) Bias (HTML)
  • Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings
  • Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
  • CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
  • StereoSet: Measuring stereotypical bias in pretrained language models
  • Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
  • Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
  • Evaluating and Mitigating Discrimination in Language Model Decisions
  • BBQ: A hand-built bias benchmark for question answering
  • Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities
  • BiasCause: Evaluate Socially Biased Causal Reasoning of Large Language Models
  • Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• 15. Bias and Fairness - III: Mitigation
  Lecture date: September 21, 2026
  tl;dr: Debiasing Techniques, Limitations of Existing Approaches, Role of Policy
  [ slides ]
  
• 16. Watermarking - I
  Lecture date: September 24, 2026
  tl;dr: Introduction, Properties of Good Watermarks, Formal Problem Setup, Red-Green Watermark
  [ slides ]
  
• 17. Watermarking - II
  Lecture date: October 05, 2026
  tl;dr: Watermarking during Training and Sampling, Threat Models
  [ slides ]
  
• 18. Uncertainty in LLMs - I: Trustworthy Confidence
  Lecture date: October 08, 2026
  tl;dr: Knowing When LLMs Should Not Be Trusted
  [ slides ]
  
• 19. Uncertainty in LLMs - II: Semantic and Claim-Level Risk
  Lecture date: October 12, 2026
  tl;dr: Unsupported Claims and Semantic Uncertainty, Claim-level Risk Detection
  [ slides ]
  
• 20. Uncertainty in LLMs - III: Safe Non-Answering
  Lecture date: October 15, 2026
  tl;dr: Abstention, Selective Generation, and Risk-Controlled Decisions
  [ slides ]
  
• 21. Uncertainty in LLMs - IV: Accountable LLM Systems
  Lecture date: October 19, 2026
  tl;dr: Evidence Grounding, Oversight, and Auditability
  [ slides ]
  
• 22. Hallucination in LLMs - I: Why LLMs Hallucinate?
  Lecture date: October 22, 2026
  tl;dr: Causes of Hallucination: Data, Training, and Inference
  [ slides ]