See also: CS 376 Schedule
Any content in the future should be considered tentative and subject to change.
Week 1: Introduction
Getting started with ML: running Python in notebooks, training an image classifier using off-the-shelf code.
Key Questions
- What is a model architecture? What is training data? What is validation data?
- How do we measure if a model is learning (accuracy vs loss)?
Objectives
- Write and execute basic Python code in Jupyter Notebooks
- Train an ML model using off-the-shelf code
- Describe the basic ML workflow (instantiating a model architecture, feeding it training data, evaluating how it generalizes)
- Understand the PPP grading system
Wednesday
- Welcome discussion: hopes and concerns
- Course logistics
- Slides: Welcome to CS 375
Friday
- Notebook: Jupyter Notebook Warmup
(name:
u01n0-notebook-warmup.ipynb; show preview, open in Colab) - Notebook: Train a simple image classifier
(name:
u01n1-train-clf.ipynb; show preview, open in Colab) - PPP credit system
- Intro to Perusall
Week 2: Array Programming & Regression
Introduction to numerical computing with NumPy/PyTorch: element-wise operations, reductions, dot products, MSE. First taste of sklearn regression.
Key Questions
- How do we represent data as arrays/tensors?
- What is a dot product and how is it used in ML?
- What does it mean to “fit” a model?
Objectives
- Implement basic essential array-computing operations (element-wise operations, reductions, dot products, MSE)
- Contrast different types of learning machines (supervised learning, unsupervised learning, RL)
- Use the sklearn API for basic regression tasks
Monday
- Lab 1 review
- Slides: Computing
- Translation workflow discussion
- Cognition required for imitation
Wednesday
- Intro to array programming
- Slides: Computing
- Notebook: PyTorch Warmup
(name:
u02n1-pytorch.ipynb; show preview, open in Colab) - Context discussion: Future of Work
Friday
- Landscape of AI/ML (supervised, unsupervised, RL)
- Slides: Learning Machines
- Notebook: Regression in
scikit-learn(name:u02n2-sklearn-regression.ipynb; show preview, open in Colab)
Week 3: LLM APIs & Classification
Using LLM APIs to build AI-powered applications. Introduction to classification models and metrics.
Key Questions
- How do we call an LLM API?
- What’s the difference between regression and classification?
- How do we evaluate a classification model (accuracy, precision, recall)?
Objectives
- Use an LLM API to build an AI-powered application
- Fit a linear regression model “by hand” using numerical computing primitives
- Describe two different ways of measuring how good a classification is
- Use basic linear and tree models for classification with sklearn
- Quantify classifier performance using accuracy, precision, recall, and cross-entropy
Monday
- Lab recap: PyTorch and sklearn notebooks
- LLM API intro: “use an AI to make an AI”
Wednesday
- Notebook: Linear Regression the Hard Way
(name:
u03n1-linreg-manual.ipynb; show preview, open in Colab) - Context discussion: AI fairness and bias
Friday
- Slides: Learning Machines
- Notebook: Classification in
scikit-learn(name:u03n2-sklearn-classification.ipynb; show preview, open in Colab) - Classification metrics (accuracy, precision, recall, cross-entropy)
Week 4: Multi-input Models & Softmax
Extending linear models to multiple inputs. Understanding softmax and cross-entropy loss.
Key Questions
- How does linear regression extend to multiple input features?
- What is softmax and why do we use it for classification?
- What is cross-entropy loss?
Objectives
- Extend understanding of linear regression to multiple input features (thinking through shapes)
- Explain the purpose and mathematical properties of the softmax operation
- Practice logistic regression and learn how to fit simple models in PyTorch
- Describe and compute cross-entropy loss
Monday
- Notebook: Multiple Linear Regression, the Hard Way
(name:
u04n1-multi-linreg-manual.ipynb; show preview, open in Colab) - Multilingual chat application demo
Wednesday
- Slides: Computing
- Notebook: Softmax, part 1
(name:
u04n2-softmax.ipynb; show preview, open in Colab) - Interactive softmax demo
Friday
- Notebook: From Linear Regression in NumPy to Logistic Regression in PyTorch
(name:
u04n3-logreg-pytorch.ipynb; show preview, open in Colab) - Hw2 soft-due: demo an AI-powered application
Week 5: Features & Review
Understanding feature extraction with ReLU. Introduction to classifier heads and bodies. Review and preparation for gradient descent.
Key Questions
- Why are good features important for neural networks?
- What is a classifier “head” vs “body”?
- How does ReLU create useful features?
Objectives
- Explain the importance of good features for neural network models
- Understand how a nonlinearity (like ReLU) can be useful for feature extraction
- Trace the data flow through an MLP, especially the shapes
- Write PyTorch expressions for each of the MLP components
- Explain the role of nonlinearities in a neural network (e.g., why they are used between linear layers)
Monday
- Feature extractors intro
- ReLU features intro
- Classifier head and body intro
Wednesday
- Notebook: ReLU Regression Interactive
(name:
u05n00-relu.ipynb; show preview, open in Colab) - Notebook: Logistic Regression and MLP
(name:
u05n2-logreg-mlp.ipynb; show preview, open in Colab) - MLP shapes practice
Friday
- Preview of learning by gradient descent
- Review day: gradient game
- Tech presentation
Week 6: Gradient Descent & Generalization
Learning by gradient descent. Understanding why generalization matters and how to measure/improve it.
Key Questions
- How does gradient descent work?
- What is overfitting vs underfitting?
- How can data augmentation help generalization?
Objectives
- Train an MLP classifier by gradient descent and know what everything does
- Describe the overall approach of Stochastic Gradient Descent: how it uses information from a batch of data to improve performance
- Describe a few ways that an ML model might do well on its training data but fail to generalize
- Describe ways to make a model generalize better (more data, data augmentation, regularization)
- Explain the importance of evaluating models on unseen data
Monday
- Gradient game activity
- Notebook: MNIST with PyTorch
(name:
u06n1-mnist-torch.ipynb; show preview, open in Colab)
Wednesday
- Notebook: Compute gradients using PyTorch
(name:
u06n2-compute-grad-pytorch.ipynb; show preview, open in Colab) - Review training loops, SGD, MLP model
Friday
- Will It Generalize? slides
- Data augmentation notebook
- Tech presentation
Week 7: Embeddings & RL
Embeddings as the data structures of neural computation. Introduction to reinforcement learning.
Key Questions
- What are embeddings and how are they used in ML?
- How does reinforcement learning differ from supervised learning?
- What is the difference between learning to mimic vs learning by exploring?
Objectives
- Explain what embeddings are (the data structures used in ML) and how they represent similarity
- Interpret vectors as points in a space and use dot product to measure similarity between data items
- Explain how a pretrained model can be repurposed for a new task by separating it into a feature extractor (body) and task-specific classifier (head)
- Describe the key differences between supervised learning and reinforcement learning
- Understand delayed rewards and exploration in RL
- Explain the difference between learning to mimic vs learning by exploring
Monday
- Embeddings Day: words, sentences, images
- Slides: Computing
- Notebook: Probe an Image Classifier
(name:
u07n1-image-embeddings.ipynb; show preview, open in Colab)
Wednesday
- Reinforcement Learning intro
- Notebook: A Reinforcement Learning Example
(name:
u07n2-rl.ipynb; show preview, open in Colab) - Optional: Notebook: u07n1-image-ops.ipynb
Friday
- Slides: CS 375: Wrap-Up
- Learning to Mimic vs Learning by Exploring
- Course wrap-up