Warning: This content has not yet been fully revised for this year.
Readings in Perusall:
- Deep Learning with Python chapter 5
Deep Learning with Python chapter 6 would also be good to read this week, but it’s not in Perusall.
Also work through the following interactive tutorials:
Supplemental Materials
A nice intuition about why layers matter: Why depth matters in a neural network (Deep Learning / AI) - YouTube
Strategies for getting state-of-the-art performance:
- Simple considerations for simple people building fancy neural networks
- How to be an effective Deep Learning Researcher/Engineer
I sometimes remark that “machine learning is lazy” (in that it tends to focus on superficial easy features). Here’s a more precise statement and study Neural Networks Learn Statistics of Increasing Complexity See also:
- What do deep networks learn and when do they learn it
- “deep over-parameterized networks are biased to find low effective rank solutions” The Low-Rank Simplicity Bias in Deep Networks | Abstract