Link Search Menu Expand Document
USC CSCI 444 Fall 2025

CSCI 444 Fall 2025: NLP

🍂 Fall 2025     ⏰ Mon / Wed 10:00 - 11:50a     📍 WPH 106

Instructor: Swabha Swayamdipta

swabhas@usc.edu

Office Hours: Monday 9-9:45am, GCS LL2 SB5

Announcements

See Brightspace.

Summary

This class is all about language models: the fundamentals, spanning from simple architectures to modern Transformer-based neural architectures underlying large language models.

Calendar + Syllabus

Week Date Class Topics Readings Work Due
1 Aug 25 Course Overview and n-grams J&M, Chap 3;
Aug 27 n-gram Models contd. J&M, Chap 3;
2 Sep 1 Labor Day HW1 Released
Sep 3 Logistic Regression J&M, Chap 4;
3 Sep 8 Project Pitches Quiz 1;
Sep 10 Word Embeddings J&M, Chap 5;
4 Sep 15 word2vec J&M, Chap 5; word2vec Explained; Group Formation Deadline;
Sep 17 Feedforward Nets J&M, Chap 6; HW1 Due
5 Sep 22 Backpropagation and Recurrent Neural Networks J&M, Chap 6; J&M, Chap 13; HW2 Released
Sep 24 Recurrent Neural Nets J&M, Chap 13; Project Proposal Due;
6 Sep 29 Seq2Seq and Attention J&M, Chap 8;
Oct 1 Transformers - Building Blocks J&M, Chap 8; Quiz 2;
7 Oct 6 PyTorch for Transformers (Lecture by Johnny Wei)
Oct 8 Fall Break
8 Oct 13 Transformer Language Models J&M Chap 8;
Oct 15 Tokenization J&M, Chap 2.5; HW2 Due
9 Oct 20 Flipped Classroom - Project Discussions HW3 Released on Oct 21
Oct 22 Language Generation J&M, Chap 7.4; J&M, Chap 12.4; Quiz 3;
10 Oct 27 Language Generation (contd.) J&M, Chap 7.4 + 7.6; Nucleus Sampling;
Oct 29 Pre-Training LLMs J&M, Chap 7; The Llama 3 Herd of Models; OLMo 2.0; Switch Transformers; Project Status Report Due;
11 Nov 3 Post-Training LLMs J&M, Chap 9; Quiz 4;
Nov 5 Paper Discussions I LoRa; InstructGPT; DPO;
12 Nov 10 Evaluating LLMs J&M, Chap 7.6; Holistic Evaluation of Language Models; MAUVE;
Nov 12 Paper Discussion II MMLU; Dynabench; ChatBot Arena;
13 Nov 17 Responsible Language Modeling J&M, Chap 7.7; HW3 Due
Nov 19 Paper Discussion III Red-Teaming; Adversarial Attacks on Aligned LMs; Generative AI and Critical Thinking;
14 Nov 24 Miscellaneous Topics and Outro Quiz 5;
Nov 26 Thanksgiving
15 Dec 1 Project Presentations I
Dec 3 Project Presentations II
16 Dec 8 Study Week
Dec 10 Study Week
17 Dec 15 Project Final Report due by 10:00am;

This calendar is subject to change. More details, e.g. lecture slides will be added as the semester continues. All work (except the project final report) is due on the specified date by 11:59 PM PT. See the syllabus for more details.

Assignments and Grading

There will be three components to course grades:

  • Homeworks (15%).
    • 5% X 3: There will be three coding homework assignments based on the topics of the class.
  • Quizzes (15%).
    • 3% X 5: Multiple-Choice Questions and Short Answers. Missed quizzes will receive a zero grade, and there will be no make-up quizzes.
  • Class Projects (55%).
    • Each student will do a group class project based on the topics covered in the class. Students will propose their own project, do the research and build a proof-of-concept, create a demonstration (e.g. video) of the proof-of-concept, and present the project in their report.
    • Pitch: 5%
    • Proposal: 10%
    • Status Reports: 10%
    • Project Presentation: 15%
    • Final Write-up: 15%
  • Paper Discussions (10%).
    • Based on student feedback, we will now have 3 classes full of paper discussions, where we will discuss a total of 9 papers on advanced topics. Students will be graded on their ability to summarize and critique papers.
  • Class Participation (5%)
    • Each student’s engagements in course discussions during class and during project discussions.

Grading inquiries and questions about the grading of the homeworks and the quizzes can be asked (to the instructor) within one week from the grading date (the date the grades are released). Grades will be available within 2-2.5 weeks after submission.

All written assignments related to the final project should use the standard *ACL paper submission template.

Late Days

Students are allowed a maximum of 6 late days total for all assignments (but NOT the quizzes or presentations). You may use up to 3 late days per assignment. Using one late day for a project assignment involves each of the teammates using a late day each. Partial late days are not permitted. For every extra late day beyond the allowed late days, the student / team will lose 20% of the grade for the assignment.

Note: Please familiarize yourself with the academic policies and read the note about student well-being.

Pre-Requisites

Students are required to have taken

  • CSCI 170 and
  • 1 from (CSCI 104 or CSCI 114) and
  • 1 from (MATH 225 or EE 141) and
  • 1 from (EE 364 or MATH 407 or BUAD 310 or ISE 225) Recommended Preparation: Fluency with Python programming on the level of ITP 216 or TAC 216

Similar Classes