CS6140 Machine Learning

HW6 - Sequence NNets for Machine Translation: Recurrent NN, LSTM, Tranformer

Make sure you check the syllabus for the due date.

To setup: see the README, get the data, get the code : training, data_utils, evaluate. In all problems below you will train and evaluate for translation from English to Spanish

PROBLEM 1 Recurrent NN : implement Encoder and Decoder [50 points]

Starting Code for Pb1
(A) Code up the Encoder and Decoder, and use the "nn.RNN" built in pipeline.
(B) Replace "nn.RNN" with the "nn.LSTM" built-in pipeline

PROBLEM 2 Recurrent NN : Implement the RNN and LSTM

Starting Code for Pb2
(A) [50 points] Encoder, Decoder: Replace "nn.RNN" with your own RNN pipeline, using code from the d2l book
(B) [optional, no credit] Encoder, Decoder: Replace "nn.LSTM" with your own LSTM, using LSTM code from the d2l book

PROBLEM 3 Transformer NN [50 points]

Starting Code for Pb3
Complete the code using the "nn.TransformerEncodinglayer" and "nn.TransformerDecodingLayer" built-in blocks

PROBLEM 4 Transformer NN : Build your own blocks [optional, no credit]

Starting Code for Pb4
Code your own EncodingBlock, DecodingBlock, and Transformer

PROBLEM 5 [optional, no credit] Adv NN : Embedding Words layer

Replace the "nn.Embedding" with your own, based on d2l book chapters 15, 16. Use Glove6B pre-trained vectors (English and Spanish) and fine-tune them for the current data.