CMSC 473/673 - Schedule

The lecture schedule will be updated as the term progresses.

Lecture Topic	Readings for this Lesson	Homework Due
What class is this? [slides]
What is NLP? [slides]	Jacob Eisenstein, NLP Chapter 1
Waitlist Deadline
Examples of NLP Tasks [slides]	Dan Jurafsky and James H. Martin, SLP Chapter 2 Jacob Eisenstein, NLP Chapter 2.2 & 4.5
Waitlists Deactivated
Examples of NLP Tasks [slides]
Last Day to Change Schedule
Machine Learning Basics & Evaluation [slides]	Jacob Eisenstein, NLP Chapter 4.3 & 4.4 Dan Jurafsky and James H. Martin, SLP Chapter 4.7, 4.8, & 4.10
Machine Learning Basics & Evaluation [slides]
		Project Milestone 1: Project Proposal
PyTorch & HuggingFace Tutorial [slides]

		HW1: Being up to the Task
Classification with Logistic Regression [slides]	Jacob Eisenstein, NLP Chapter 2.5 Dan Jurafsky and James H. Martin, NLP Chapter 5.1-5.3 (+intro)
Learning & Using Logistic Regression [slides]	Jacob Eisenstein, NLP Chapter B.1 Dan Jurafsky and James H. Martin, NLP Chapter 5.4-5.11
Vector Embeddings [slides]	Jacob Eisenstein, NLP Chapter 14 Dan Jurafsky and James H. Martin, SLP Chapter 6	Grad Checkpoint 1: Selection
Vector Embeddings [slides]
No Class - Project Milestone Due		Project Milestone 2: Literature Review
Spring Break - No Class
Spring Break - No Class
N-Gram Language Models [slides]	Jacob Eisenstein, NLP Chapter 6.1 Dan Jurafsky and James H. Martin, SLP Chapter 3
N-Gram Language Models [slides]
Neural Language Models [slides]	Dan Jurafsky and James H. Martin, SLP Chapter 7
Recurrent Neural Network LMs [slides]	Jacob Eisenstein, NLP Chapter 6.3 & 18.3 (intro only) Dan Jurafsky and James H. Martin, SLP Chapter 9	Project Milestone 3: Halfway Checkpoint
Last Day to Withdraw from Individual Classes
Recurrent Neural Network LMs Tutorial [slides]		HW 2: Evaluation
Attention & Transformers [slides]	Jacob Eisenstein, NLP Chapter 18.3.1-2 Ashish Vaswani, et al., Attention Is All You Need
Decoding, Sampling, & Transformer Fine-tuning [slides]	Dan Jurafsky and James H. Martin, SLP Chapter 11.3
Types of Foundational Models [slides]	Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Alec Radford, et al., Language Models are Unsupervised Multitask Learners Dan Jurafsky and James H. Martin, SLP Chapter 11.1-11.2 (+intro)
Ethics of NLP [slides]	Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 Roy Schwartz, Jesse Dodge, Noah A. Smith, Oren Etzioni, Green AI Daphne Ippolito, et al., Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy Vered Shwartz, Yejin Choi, Do Neural Language Models Overcome Reporting Bias? Xuhui Zhou, et al., Challenges in Automated Debiasing for Toxic Language Detection
Prompt Engineering Tutorial [slides]	Tom B. Brown, et al., Language Models are Few-Shot Learners Jason Wei, et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models	Grad Checkpoint 2: Draft

Alignment [slides]	Long Ouyang, et al., Training language models to follow instructions with human feedback	Grad Checkpoint 3: Feedback
No Class - Project Milestone Due		Project Milestone 4: Rough Draft
Applications: Machine Translation [slides]	Dan Jurafsky and James H. Martin, SLP Chapter 13: Machine Translation	HW 3: Prompt Engineering
Applications: Dialog Systems [slides]	Jacob Eisenstein, NLP Chapter 19.3: Dialogue Dan Jurafsky and James H. Martin, SLP Chapter 15: Chatbots and Dialogue Systems
Applications: Creative Generation	Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein, Re3: Generating Longer Stories With Recursive Reprompting and Revision
Project Presentations, 10:30 AM-12:30 PM		Project Milestone 5: Final Submission Grad Checkpoint 4: Final Submission