This repository consists of cource Standford CS224n: Natural Language Processing with Deep Learning on Winter 2021. And Also include 2018 CS224n because of assignment 5 related to Convolution model based on pytorch and Colab(.ipynb)
Course Related Links
2021/11/04 - This contains every assignment 1 to assignment 5 in 2018 and 2021. And also there are flowchart & codeflow at each steps for attention, RNN, and transformer model in assignment 4 and 5. These contents are written to blog. These contents are referred to CS224n.
But it has some limitation, which is written in the blog. At least all of assignments can run in Colab and categorized Concepts. If you want to add more, please commit on this repository or comment.
*Blog page is preparing...
It's based on the blog pages. All contents mentions the reference if it comes from other blog.
[O] WEEK 1 What is the basic concept and Framework in Natural Language Process?
[O] WEEK 2 How to Calculate in Neural Network and What is the Dependency Parsing?
[O] WEEK 3 How to struct the Dependency Parsing and the Problem solved by RNN, Bi-RNN, GRU, LTSM?
[O] WEEK 4 Machine Translation, Sequence-to-Sequence And Attention
[O] WEEK 5 Self-Attention and Transformers
After WEEK 5, lectures explain
[ ] QA(Week 5-6)
[ ] Generation(Week 6)
[ ] Coreference Resolution(Week 7)
[ ] T5(Week 7)
[ ] Language Model(Week 8)
[ ] Ethics(Week 8)
[ ] Analysis and Explanation(Week 9)
[ ] Future(Week 9)
But I want to learn more closer to contents related to Speech, so I completed this lecture and continue to CS224S
It was based on Pytorch and used the GPU on Colab.
References:
- https://github.com/pcyin/pytorch_nmt
- https://gitlab.com/vojtamolda/stanford-cs224n-nlp-with-dl/-/tree/master/
- https://github.com/ZubinGou/CS224n-Assignment
The contents is belows. It contains code and written part that all parts were completed but 2018 Assignment 5. 1-e wasn't finished. And Assignment 4. NMT model has lower than the cut line, which mentioned in blog(Preparing...) TODO category. It will be improved in the future.
[O] Assignment 1. Introduction to word vectors
[O] Assignment 2. Derivatives and implementation of word2vec algorithm
[O] Assignment 3. Dependency Parsing
[O] Assignment 4. NMT model
[O] [2018] Assignment 5 Character-based Convolutional NMT
[O] [2021] Assignment 5 Self-Attention, Transformers, and Pretraining
And each of code also contains comments for understanding tensor shape in each step, and In Assignment 4, Assignment 5, there are flowchart & codeflow for each model. A little contents were written by Korean but you cant easily read because main contents were flow.
It will be improved in the future, which is for speech recognition contents.