Skip to content

Commit e2e1c3e

Browse files
jeremymanningjeremymanning
committed
Restructure course: combine embeddings lectures 9-11 into 9-10, renumber 12-27 to 11-26
- Move lecture 8 (POS/sentiment) from week2 to week3 Wednesday - Combine old lectures 9, 10, 11 (93 slides) into streamlined lectures 9-10: - New lecture 9: Classic Embeddings (LSA/LDA) - 19 slides - New lecture 10: Word Embeddings (Word2Vec, GloVe, FastText) - 21 slides - Renumber all subsequent lectures (12→11 through 27→26) - Fix slide cut-off issues with scale directives in lectures 9 and 10 - Update syllabus.md and README.md to reflect new schedule
1 parent a6e0da8 commit e2e1c3e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+24931
-29273
lines changed

‎admin/syllabus.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -163,49 +163,49 @@ We strive to create an inclusive learning environment where all students feel su
163163
|-----|---------|--------|--------|-----------|
164164
| Mon Jan 12 | Lecture 5 | Data Cleaning & Preprocessing | [HTML](../slides/week2/lecture5.html), [PDF](../slides/week2/lecture5.pdf) | HuggingFace NLP Course Chapter 2 |
165165
| Wed Jan 14 | Lecture 6 | Tokenization | [HTML](../slides/week2/lecture6.html), [PDF](../slides/week2/lecture6.pdf) | [Sennrich et al. (2016)](https://aclanthology.org/P16-1162/), [Kudo & Richardson (2018)](https://aclanthology.org/D18-2012/) |
166-
| Thu Jan 15 | Lecture 7 | Text Classification Workshop | [HTML](../slides/week2/lecture7.html), [PDF](../slides/week2/lecture7.pdf) | |
167-
| Fri Jan 16 | Lecture 8 | POS Tagging & Sentiment Analysis | [HTML](../slides/week2/lecture8.html), [PDF](../slides/week2/lecture8.pdf) | **[Assignment 2](../assignments/assignment-2/) Released**, **[Assignment 1](../assignments/assignment-1/) Due** |
166+
| Thu Jan 15 | Lecture 7 (X-hour) | Text Classification Workshop | [HTML](../slides/week2/lecture7.html), [PDF](../slides/week2/lecture7.pdf) | |
167+
| Fri Jan 16 | Lecture 7 (continued) | Text Classification Workshop | [HTML](../slides/week2/lecture7.html), [PDF](../slides/week2/lecture7.pdf) | **[Assignment 2](../assignments/assignment-2/) Released**, **[Assignment 1](../assignments/assignment-1/) Due** |
168168

169169
### Week 3: Text Embeddings I (January 19--23)
170170

171171
| Day | Lecture | Topics | Slides | Materials |
172172
|-----|---------|--------|--------|-----------|
173173
| Mon Jan 19 | -- | Martin Luther King Jr. Day (No Class) | -- | |
174-
| Wed Jan 21 | Lecture 9 | Classic Embeddings (LSA, LDA) | [HTML](../slides/week3/lecture9.html), [PDF](../slides/week3/lecture9.pdf) | [Landauer & Dumais (1997)](https://psycnet.apa.org/record/1997-02478-006), [Blei et al. (2003)](https://www.jmlr.org/papers/v3/blei03a) |
175-
| Thu Jan 22 | Lecture 10 | Embeddings Workshop | [HTML](../slides/week3/lecture10.html), [PDF](../slides/week3/lecture10.pdf) | |
176-
| Fri Jan 23 | Lecture 11 | Word Embeddings (Word2Vec, GloVe) | [HTML](../slides/week3/lecture11.html), [PDF](../slides/week3/lecture11.pdf) | [Mikolov et al. (2013)](https://arxiv.org/abs/1301.3781), [Pennington et al. (2014)](https://aclanthology.org/D14-1162/), **[Assignment 3](../assignments/assignment-3/) Released**, **[Assignment 2](../assignments/assignment-2/) Due** |
174+
| Wed Jan 21 | Lecture 8 | POS Tagging & Sentiment Analysis | [HTML](../slides/week3/lecture8.html), [PDF](../slides/week3/lecture8.pdf) | [Linzen et al. (2016)](https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00115/1567418/tacl_a_00115.pdf) |
175+
| Thu Jan 22 | Lecture 9 (X-hour) | Classic Embeddings (LSA, LDA) | [HTML](../slides/week3/lecture9.html), [PDF](../slides/week3/lecture9.pdf) | [Deerwester et al. (1990)](http://wordvec.colorado.edu/papers/Deerwester_1990.pdf), [Blei et al. (2003)](https://www.jmlr.org/papers/v3/blei03a) |
176+
| Fri Jan 23 | Lecture 10 | Word Embeddings (Word2Vec, GloVe, FastText) | [HTML](../slides/week3/lecture10.html), [PDF](../slides/week3/lecture10.pdf) | [Mikolov et al. (2013)](https://arxiv.org/abs/1301.3781), [Pennington et al. (2014)](https://aclanthology.org/D14-1162/), **[Assignment 3](../assignments/assignment-3/) Released**, **[Assignment 2](../assignments/assignment-2/) Due** |
177177

178178
### Week 4: Text Embeddings II (January 26--30)
179179

180180
| Day | Lecture | Topics | Slides | Materials |
181181
|-----|---------|--------|--------|-----------|
182-
| Mon Jan 26 | Lecture 12 | Contextual Embeddings | [HTML](../slides/week4/lecture12.html), [PDF](../slides/week4/lecture12.pdf) | [Peters et al. (2018)](https://aclanthology.org/N18-1202/), [Cer et al. (2018)](https://arxiv.org/abs/1803.11175) |
183-
| Wed Jan 28 | Lecture 13 | Dimensionality Reduction | [HTML](../slides/week4/lecture13.html), [PDF](../slides/week4/lecture13.pdf) | [McInnes et al. (2018)](https://arxiv.org/abs/1802.03426) |
184-
| Fri Jan 30 | Lecture 14 | Cognitive Models of Semantic Representation | [HTML](../slides/week4/lecture14.html), [PDF](../slides/week4/lecture14.pdf) | [Anderson et al. (2016)](https://www.jneurosci.org/content/36/45/11444), **[Assignment 4](../assignments/assignment-4/) Released**, **[Assignment 3](../assignments/assignment-3/) Due (Feb 2, 11:59 PM EST)** |
182+
| Mon Jan 26 | Lecture 11 | Contextual Embeddings | [HTML](../slides/week4/lecture11.html), [PDF](../slides/week4/lecture11.pdf) | [Peters et al. (2018)](https://aclanthology.org/N18-1202/), [Cer et al. (2018)](https://arxiv.org/abs/1803.11175) |
183+
| Wed Jan 28 | Lecture 12 | Dimensionality Reduction | [HTML](../slides/week4/lecture12.html), [PDF](../slides/week4/lecture12.pdf) | [McInnes et al. (2018)](https://arxiv.org/abs/1802.03426) |
184+
| Fri Jan 30 | Lecture 13 | Cognitive Models of Semantic Representation | [HTML](../slides/week4/lecture13.html), [PDF](../slides/week4/lecture13.pdf) | [Anderson et al. (2016)](https://www.jneurosci.org/content/36/45/11444), **[Assignment 4](../assignments/assignment-4/) Released**, **[Assignment 3](../assignments/assignment-3/) Due (Feb 2, 11:59 PM EST)** |
185185

186186
### Week 5: Transformers & Attention (February 2--6)
187187

188188
| Day | Lecture | Topics | Slides | Materials |
189189
|-----|---------|--------|--------|-----------|
190-
| Mon Feb 2 | Lecture 15 | Attention Mechanisms | [HTML](../slides/week5/lecture15.html), [PDF](../slides/week5/lecture15.pdf) | [Bahdanau et al. (2015)](https://arxiv.org/abs/1409.0473), [Vaswani et al. (2017)](https://arxiv.org/abs/1706.03762) |
191-
| Wed Feb 4 | Lecture 16 | Transformer Architecture | [HTML](../slides/week5/lecture16.html), [PDF](../slides/week5/lecture16.pdf) | [Vaswani et al. (2017)](https://arxiv.org/abs/1706.03762), HuggingFace NLP Course Chapter 3 |
192-
| Thu Feb 6 | Lecture 17 | Training Transformers | [HTML](../slides/week5/lecture17.html), [PDF](../slides/week5/lecture17.pdf) | **[Assignment 4](../assignments/assignment-4/) Due (Feb 9, 11:59 PM EST)** |
190+
| Mon Feb 2 | Lecture 14 | Attention Mechanisms | [HTML](../slides/week5/lecture14.html), [PDF](../slides/week5/lecture14.pdf) | [Bahdanau et al. (2015)](https://arxiv.org/abs/1409.0473), [Vaswani et al. (2017)](https://arxiv.org/abs/1706.03762) |
191+
| Wed Feb 4 | Lecture 15 | Transformer Architecture | [HTML](../slides/week5/lecture15.html), [PDF](../slides/week5/lecture15.pdf) | [Vaswani et al. (2017)](https://arxiv.org/abs/1706.03762), HuggingFace NLP Course Chapter 3 |
192+
| Fri Feb 6 | Lecture 16 | Training Transformers | [HTML](../slides/week5/lecture16.html), [PDF](../slides/week5/lecture16.pdf) | **[Assignment 4](../assignments/assignment-4/) Due (Feb 9, 11:59 PM EST)** |
193193

194194
### Week 6: Encoder Models (February 9--13)
195195

196196
| Day | Lecture | Topics | Slides | Materials |
197197
|-----|---------|--------|--------|-----------|
198-
| Mon Feb 9 | Lecture 18 | BERT Deep Dive | [HTML](../slides/week6/lecture18.html), [PDF](../slides/week6/lecture18.pdf) | [Devlin et al. (2019)](https://aclanthology.org/N19-1423/), HuggingFace NLP Course Chapter 4, **[Assignment 5](../assignments/assignment-5/) Released** |
199-
| Wed Feb 11 | Lecture 19 | BERT Variants | [HTML](../slides/week6/lecture19.html), [PDF](../slides/week6/lecture19.pdf) | [Liu et al. (2019)](https://arxiv.org/abs/1907.11692), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) |
200-
| Thu Feb 13 | Lecture 20 | Applications of Encoder Models | [HTML](../slides/week6/lecture20.html), [PDF](../slides/week6/lecture20.pdf) | **[Assignment 5](../assignments/assignment-5/) Due (Feb 16, 11:59 PM EST)** |
198+
| Mon Feb 9 | Lecture 17 | BERT Deep Dive | [HTML](../slides/week6/lecture17.html), [PDF](../slides/week6/lecture17.pdf) | [Devlin et al. (2019)](https://aclanthology.org/N19-1423/), HuggingFace NLP Course Chapter 4, **[Assignment 5](../assignments/assignment-5/) Released** |
199+
| Wed Feb 11 | Lecture 18 | BERT Variants | [HTML](../slides/week6/lecture18.html), [PDF](../slides/week6/lecture18.pdf) | [Liu et al. (2019)](https://arxiv.org/abs/1907.11692), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) |
200+
| Fri Feb 13 | Lecture 19 | Applications of Encoder Models | [HTML](../slides/week6/lecture19.html), [PDF](../slides/week6/lecture19.pdf) | **[Assignment 5](../assignments/assignment-5/) Due (Feb 16, 11:59 PM EST)** |
201201

202202
### Week 7: Decoder Models & GPT (February 16--20)
203203

204204
| Day | Lecture | Topics | Slides | Materials |
205205
|-----|---------|--------|--------|-----------|
206-
| Mon Feb 16 | Lecture 21 | GPT Architecture | [HTML](../slides/week7/lecture21.html), [PDF](../slides/week7/lecture21.pdf) | [Radford et al. (2018)](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), [Radford et al. (2019)](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) |
207-
| Wed Feb 18 | Lecture 22 | Scaling Up to GPT-3 and Beyond | [HTML](../slides/week7/lecture22.html), [PDF](../slides/week7/lecture22.pdf) | [Brown et al. (2020)](https://arxiv.org/abs/2005.14165), [OpenAI (2023)](https://arxiv.org/abs/2303.08774) |
208-
| Fri Feb 20 | Lecture 23 | Implementing GPT from Scratch | [HTML](../slides/week7/lecture23.html), [PDF](../slides/week7/lecture23.pdf) | |
206+
| Mon Feb 16 | Lecture 20 | GPT Architecture | [HTML](../slides/week7/lecture20.html), [PDF](../slides/week7/lecture20.pdf) | [Radford et al. (2018)](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), [Radford et al. (2019)](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) |
207+
| Wed Feb 18 | Lecture 21 | Scaling Up to GPT-3 and Beyond | [HTML](../slides/week7/lecture21.html), [PDF](../slides/week7/lecture21.pdf) | [Brown et al. (2020)](https://arxiv.org/abs/2005.14165), [OpenAI (2023)](https://arxiv.org/abs/2303.08774) |
208+
| Fri Feb 20 | Lecture 22 | Implementing GPT from Scratch | [HTML](../slides/week7/lecture22.html), [PDF](../slides/week7/lecture22.pdf) | |
209209

210210
### Week 8: No Classes (February 23--27)
211211

@@ -215,15 +215,15 @@ Instructor away February 23--27. No classes this week. Use this time to work on
215215

216216
| Day | Lecture | Topics | Slides | Materials |
217217
|-----|---------|--------|--------|-----------|
218-
| Mon Mar 2 | Lecture 24 | Retrieval Augmented Generation (RAG) | [HTML](../slides/week9/lecture24.html), [PDF](../slides/week9/lecture24.pdf) | [Lewis et al. (2020)](https://arxiv.org/abs/2005.11401) |
219-
| Wed Mar 4 | Lecture 25 | Mixture of Experts & Efficiency | [HTML](../slides/week9/lecture25.html), [PDF](../slides/week9/lecture25.pdf) | [Fedus et al. (2022)](https://arxiv.org/abs/2101.03961), [Jiang et al. (2024)](https://arxiv.org/abs/2401.04088) |
220-
| Fri Mar 6 | Lecture 26 | Ethics, Bias, and Safety | [HTML](../slides/week9/lecture26.html), [PDF](../slides/week9/lecture26.pdf) | [Bender et al. (2021)](https://dl.acm.org/doi/10.1145/3442188.3445922) |
218+
| Mon Mar 2 | Lecture 23 | Retrieval Augmented Generation (RAG) | [HTML](../slides/week9/lecture23.html), [PDF](../slides/week9/lecture23.pdf) | [Lewis et al. (2020)](https://arxiv.org/abs/2005.11401) |
219+
| Wed Mar 4 | Lecture 24 | Mixture of Experts & Efficiency | [HTML](../slides/week9/lecture24.html), [PDF](../slides/week9/lecture24.pdf) | [Fedus et al. (2022)](https://arxiv.org/abs/2101.03961), [Jiang et al. (2024)](https://arxiv.org/abs/2401.04088) |
220+
| Fri Mar 6 | Lecture 25 | Ethics, Bias, and Safety | [HTML](../slides/week9/lecture25.html), [PDF](../slides/week9/lecture25.pdf) | [Bender et al. (2021)](https://dl.acm.org/doi/10.1145/3442188.3445922) |
221221

222222
### Week 10: Final Projects (March 9)
223223

224224
| Day | Lecture | Topics | Slides | Materials |
225225
|-----|---------|--------|--------|-----------|
226-
| Mon Mar 9 | Lecture 27 | Final Project Presentations & Wrap-up | -- | Last day of classes, **[Final Project](../assignments/final-project/) Due (Mar 9, 11:59 PM EST)** |
226+
| Mon Mar 9 | Lecture 26 | Final Project Presentations & Wrap-up | -- | Last day of classes, **[Final Project](../assignments/final-project/) Due (Mar 9, 11:59 PM EST)** |
227227

228228
### Final Exam Period (March 13--17)
229229

0 commit comments

Comments
 (0)