Advanced Large Language Models
Semester I, 2025-26 | Subscribe to Newsletter!
Updates
- New Lecture is up: 38. Interpretability of LLMs [ slides ]
- New Lecture is up: 37. Physics of LLMs: Knowledge Manipulation, Knowledge Capacity Scaling Laws and Reasoning [ slides ]
- New Lecture is up: 36. Physics of LLMs: Knowledge Storage and Extraction [ slides ]
- New Lecture is up: 35. Alternative Models: Discretized SSMs & Structured Discrete SSMs [ slides-1 ] [ slides-2 ]
- New Lecture is up: 34. Text Generation with Multimodal Inputs [ slides ]
- New Lecture is up: 33. Multimodal Encoder Models [ slides ]
- New Lecture is up: 32. Alternative Models: SSMs [ slides ]
Course Description
The field of Natural Language Processing (NLP) has witnessed rapid progress in recent times, driven mainly by the design and development of Large Language Models (LLMs). With the increase in scale, LLMs exhibit various emergent properties, though there are conflicting opinions among researchers about these phenomena. Nonetheless, LLMs are proving to be useful and are becoming ubiquitous across numerous applications.
This advanced course aims to introduce the latest advancements in generative AI for text and is open to both undergraduate and graduate students. The course is structured into five modules: Fundamentals, Efficiency, Augmentation & Reasoning, Alternate Paradigms, and Miscellaneous. Together, these modules will provide a comprehensive view of the different facets of LLMs.
Students should have a background in Machine Learning and be proficient in Python programming. At least some basic knowledge of Deep Learning and NLP is preferred. Through assignments and a course project, students will acquire the skills necessary to design, implement, and understand LLMs using the PyTorch framework.
Previous Offerings
Logistics
- Timings: Slot H (Monday, Wednesday: 11 am - 12 pm; Thursday: 12 - 1 pm)
- Office hours: Monday: 5 - 5:30 pm
- Classroom: Bharti-301
- News and announcements: All the news and announcements will be posted on Piazza.
- Piazza link: Sign up for this class on Piazza. Check your mail for the access code. Students are encouraged to ask questions and participate in discussions!
- Assignment submission: All assignments should be submitted on Moodle.
- Audit policy: To get an Audit Pass in the course, you must achieve a score equivalent to at least a B- grade or higher.
Assessment Plan (Tentative)
- Minor: 15%
- Major: 25%
- 2x Quizzes (in-class): 20%
- 1x Assignment (to be done individually): 15%
- Project (group-wise): 25%