Large Language Models (LLMs): Introduction and Recent Advances
Semester I, 2024-25 | Subscribe to Newsletter!
Updates
Course Description
The field of Natural Language Processing (NLP) has witnessed rapid progress in recent times, driven mainly by the design and development of Large Language Models (LLMs). With the increase in scale, LLMs exhibit various emergent properties, though there are conflicting opinions among researchers about these phenomena. Nonetheless, LLMs are proving to be useful and are becoming ubiquitous across numerous applications.
This advanced course aims to introduce the latest advancements in generative AI for text and is open to both undergraduate and graduate students. The course is structured into five modules: Basics, Architecture, Learnability, User Acceptability, and Ethics & Miscellaneous. Together, these modules will provide a comprehensive view of the different facets of LLMs.
Students should have a background in Machine Learning and be proficient in Python programming. At least some basic knowledge of Deep Learning and NLP is preferred. Through assignments and a course project, students will acquire the skills necessary to design, implement, and understand LLMs using the PyTorch framework.
Previous Offerings
This is a new course designed to teach the students about recent advances in the field LLMs, building gradually from the basics. While it is impossible to teach about all recent research, more so in case of a field advancing as rapidly as LLMs, we will try our best to cover the important and well-established topics.
The NLP course offered by Prof. Tanmoy is ideally a precursor to this course. We will spend more time on the advanced concepts in this course and will go into a considerable depth, unlike NLP course where more time is spent on the basics. You can consider looking into the previous offerings of the basic NLP course:
Teaching Assistant
Logistics
- Timings: Slot H (Monday, Wednesday: 11 am - 12 pm; Thursday: 12 - 1 pm)
- Office hours: Monday: 5 - 5:30 pm
- Classroom: Bharti-301
- News and announcements: All the news and announcements will be posted on Piazza.
- Piazza link: Sign up for this class on Piazza. Students are encouraged to ask questions and participate in discussions!
- Assignment submission: All assignments should be submitted on Moodle.
- YouTube link: The lecture recordings are uploaded on this YouTube channel.
Assessment Plan (Tentative)
- Minor: 15%
- Major: 25%
- 2x Quizzes (in-class): 10%
- 1x Assignment (to be done individually): 20%
- Project (group-wise): 30%