• Unit 1: Introduction
• Unit 2: Supervised Learning
• Unit 3: ML Fundamentals
• Unit 4: Neural Models
• Unit 5: Embeddings
• Unit 6: Generalization
• Unit 7: Vision and Perspectives
• 376 Unit 1: Introduction to Generative Modeling
• 376 Unit 2: Language Modeling
• 376 Unit 3: Architectures
• Neural Computation
• 376 Unit 4: Generation and Prompt Engineering
• ML Systems
• 376 Unit 5: Multimodal Models and Diffusion
• 376 Unit 6: Miscellaneous Topics
• Learning Machines
• Context and Implications
• Review

Unit 4: Neural Models

In this unit we extend our modeling skills to encompass classification models, and start to build the tools that will let us represent complex functions by using hidden layers. Both of these objectives require us to learn about nonlinear operations. We’ll focus on the two most commonly used ones: the softmax operator (which converts scores to probabilities) and the rectifier (“ReLU”, which clips negative values).

Neural Models

• Describe and compute cross-entropy loss
• Explain the purpose and mathematical properties of the softmax operation.
• Explain the role of nonlinearities in a neural network (e.g., why they are used between linear layers)
• Implement a logistic regression model using basic numerical computing primitives
• Implement numerical computing operations on multidimensional arrays using PyTorch
• Explain the purpose of automatic differentiation in machine learning

Automatic Differentiation

We’ll be doing some automatic differentiation this week:

• Yet another backpropagation tutorial by Harvard CS Professor Boaz Barak Overview of PyTorch Autograd Engine | PyTorch
• Automatic differentiation in machine learning: a survey
• autograd-for-dummies: A minimal autograd engine and neural network library for machine learning students.

Contents