Our goals this week are:
- Peer feedback on Homework 3
- Practice working with tokenization, so that you have a clear idea about how you might convert strings into sequences of numbers that you can provide to a neural net
Homework 3 Peer Feedback
Find a partner or two (ideally someone who you don’t usually work with). Swap notebooks (the easiest way might be to just trade computers). First, look at the partner’s notebook on your own, and answer the following questions:
- Looking only at the code, describe what model your partner trained. (How many layers? What dimensionality? What activation function? Were there bias terms? Was any regularization used? What learning rate was used? How did they initialize the linear layers?)
- Now look at the prose (Markdown cells). Could someone who’s not in this class understand it? Does the explanation match the code? (If not, what’s different?) Is there anything missing?
Once you’ve considered those questions on your own, rejoin your partner and consider the following questions together:
- What differences did you notice between your notebooks? What did you learn from each other?
- Which classifier got better performance on the training set? Which classifier got better performance on the validation set? Can you figure out why? Was the difference meaningful? Consider the performance in the first epoch as well as the performance after training.
- Join forces: come up with a single notebook that combines the best parts of both of your notebooks. (You’ll need to pick one person to be the “owner” of the notebook; make sure you share the results with your partner.) Try to make one tangible improvement over the original notebooks. You might:
- Try to achieve a higher accuracy
- Get a better sense of the effect of a hyperparameter, such as learning rate, hidden layer dimensionality, or whether to use bias terms.
- Make the code more readable
- Make the explanation more clear.
You can submit your improved notebook as a revision to Homework 3; it will not be counted as late.
Additionally, please complete the peer feedback assignment on Moodle, which will ask you to answer the following questions:
- What did you learn from your partner’s notebook?
- What feedback did you give your partner? Did they find it useful?
- What feedback did you receive? What did you learn from it?
This Week’s Fundamentals
There’s only one notebook this week, designed to help you practice with tokenizers. As usual, do your work in the corresponding notebook, and fill out the check-in quiz on Moodle when you’re done.
- Tokenization
(name:
u08n1-tokenization.ipynb; show preview, open in Colab)