Fundamentals Index

Goal

These notebooks will demonstrate proficiency in basic machine learning concepts and skills.

To complete a Fundamentals notebook, follow instructions and fill in blanks. Most blanks will be labeled # your code here, an ellipsis (...), or *your answer here* (for narrative answers written in Markdown). You should remove placeholder comments.

Successful solutions will:

• Include code that successfully accomplishes the task.
  • It should generate the results when run fresh (“Restart and Run All”)
  • It should have no extraneous code.
  • Format code clearly (consistent spacing, one idea per line, no overly long lines, etc.)
• Document each major step succinctly but clearly.
  • Use Markdown cells (with appropriate formatting and links) to describe the overall steps taken.
    • See the Setup section of various notebooks for an example of code explanations.
    • Note that you are not required to understand the code in the “Setup” section.
  • Use clear variable names, keyword arguments, and code comments to make the code easy to follow.
• Include responses to each of the analysis questions.
  • Add a Markdown cell for each question.
  • Add code cells as necessary to run computations that some questions may need.
  • Any activities marked “Extension” are optional but encouraged.

We aim that each notebook will:

• Demonstrate a single concept
• Take less than 15 minutes to complete, if that concept is understood (if it’s taking longer than 15 minutes, please let the instructor know so it can be simplified in the future)
• Take less than 5 minutes to run to completion
• Be a useful reference for how to perform that operation in the future

We also strive for the sequence to make sense.

The Notebooks

Note: Notebooks beyond the current unit may not be updated for the current year.

• Unit 1
  • Jupyter Notebook Warmup (name: u01n0-notebook-warmup.ipynb; show preview, open in Colab)
  • Train a simple image classifier (name: u01n1-train-clf.ipynb; show preview, open in Colab)
• Unit 2
  • PyTorch Warmup (name: u02n1-pytorch.ipynb; show preview, open in Colab)
  • Image Operations (name: u02n2-image-ops.ipynb; show preview, open in Colab)
• Unit 3
  • Classifier Diagnostics (name: u03n1-diagnose-image-classifier.ipynb; show preview, open in Colab)
• Unit 4
  • Compute gradients using PyTorch (name: u04n1-compute-grad-PyTorch.ipynb; show preview, open in Colab)
  • Linear Regression the PyTorch Way (name: u04n2-linreg-manual.ipynb; show preview, open in Colab)
• Unit 5
  • Softmax, part 1 (name: u05n1-softmax.ipynb; show preview, open in Colab)
  • Softmax and Sigmoid (name: u05n2-softmax-2.ipynb; show preview, open in Colab)
  • Train Simple Image Classifier (name: u05n3-mnist-clf.ipynb; show preview, open in Colab)
• Unit 6
  • Trace Simple Image Classifier (name: u06n1-trace-mnist.ipynb; show preview, open in Colab)
  • Linear Regression using the Fast.ai Learner (name: u06n2-linreg-learner.ipynb; show preview, open in Colab)
  • Nonlinear Regression (name: u06n3-nn-regression.ipynb; show preview, open in Colab)
• Unit 7
  • Regression in scikit-learn (name: u07n1-sklearn-regression.ipynb; show preview, open in Colab)
  • Classification in scikit-learn (name: u07n2-sklearn-classification.ipynb; show preview, open in Colab)
• Unit 8
  • Tokenization (name: u08n1-tokenization.ipynb; show preview, open in Colab)
• Unit 9
  • Logits in Causal Language Models (name: u09n1-lm-logits.ipynb; show preview, open in Colab)
  • An exercise on bias in word embeddings. (name: u09n1-word-embeddings.ipynb; show preview, open in Colab)
• Unit 11
  • Demo of Logits and Embeddings from a Language Model (name: u11n0-logits-demo.ipynb; show preview, open in Colab)
  • Translation as Language Modeling (name: u11n1-decoding.ipynb; show preview, open in Colab)
  • Programming with Self-Attention (name: u11n1-self-attention.ipynb; show preview, open in Colab)
  • Implementing self-attention (name: u11n2-implement-transformer.ipynb; show preview, open in Colab)
• Unit 12
  • Models for Sequence Data (name: u12n1-seq-models.ipynb; show preview, open in Colab)
  • Time Series Modeling and EDA (name: u12n1-time-series.ipynb; show preview, open in Colab)
• Unit 13
  • Why so big? Counting parameters in sequence models (name: u13n1-count-params.ipynb; show preview, open in Colab)
  • Stable Diffusion Deep Dive (name: u13n2-stable-diffusion.ipynb; show preview, open in Colab)