Advanced Large Language Models
Semester I, 2025-26 | Subscribe to Newsletter!
Updates
- New Lecture is up: 21. Retrieval-based LMs: Part 02 [ slides ]
- New Lecture is up: 20. Retrieval-based LMs: Part 01 [ slides ]
- New Lecture is up: 19. Knowledge Distillation [ slides ]
- New Lecture is up: 18. Model Compression [ slides ]
- New Lecture is up: 17. Parameter-Efficient Fine-Tuning (PEFT) [ slides ]
-
Piazza link: Sign up for this class on Piazza. Check your mail for the access code.
Course Description
The field of Natural Language Processing (NLP) has witnessed rapid progress in recent times, driven mainly by the design and development of Large Language Models (LLMs). With the increase in scale, LLMs exhibit various emergent properties, though there are conflicting opinions among researchers about these phenomena. Nonetheless, LLMs are proving to be useful and are becoming ubiquitous across numerous applications.
This advanced course aims to introduce the latest advancements in generative AI for text and is open to both undergraduate and graduate students. The course is structured into five modules: Fundamentals, Efficiency, Augmentation & Reasoning, Alternate Paradigms, and Miscellaneous. Together, these modules will provide a comprehensive view of the different facets of LLMs.
Students should have a background in Machine Learning and be proficient in Python programming. At least some basic knowledge of Deep Learning and NLP is preferred. Through assignments and a course project, students will acquire the skills necessary to design, implement, and understand LLMs using the PyTorch framework.
Previous Offerings
Logistics
- Timings: Slot H (Monday, Wednesday: 11 am - 12 pm; Thursday: 12 - 1 pm)
- Office hours: Monday: 5 - 5:30 pm
- Classroom: Bharti-301
- News and announcements: All the news and announcements will be posted on Piazza.
- Piazza link: Sign up for this class on Piazza. Check your mail for the access code. Students are encouraged to ask questions and participate in discussions!
- Assignment submission: All assignments should be submitted on Moodle.
- Audit policy: To get an Audit Pass in the course, you must achieve a score equivalent to at least a B- grade or higher.
Assessment Plan (Tentative)
- Minor: 15%
- Major: 25%
- 2x Quizzes (in-class): 20%
- 1x Assignment (to be done individually): 15%
- Project (group-wise): 25%