• Unit 1: Introduction
• Unit 2: Data
• Unit 3: Data and Ethics
• Unit 4: Models
• Unit 5: Learning and Classification
• Unit 6: Recap and Regularization
• Unit 7: Learned Representations (Embeddings)
• Unit 8: NLP Intro
• Unit 9: NLP Modeling
• Unit 10: Transformers
• Unit 11: Generation
• Unit 12: Review, Reinforcement Learning
• Unit 13: Human-Centered AI

Unit 12: Review, Reinforcement Learning

Preparation

• Watch MIT 6.S191 Lecture 5: Deep Reinforcement Learning: [Slides], [Video]

Supplemental Material

• Contextual
  • AlphaGo Documentary
  • ACM Selects: AI for Robotics
• Technical
  • Using Sequence Models for RL
    • Overview: Hugging Face blog post
    • Sequence Modeling Solutions for Reinforcement Learning Problems (a simple and clever approach)
      • See also: Decision Transformer: Reinforcement Learning via Sequence Modeling | Papers With Code
  • Spinning Up in Deep RL - a hands-on introduction to reinforcement learning in PyTorch by OpenAI
  • Creativity and Exploration
    • one example paper: BeBold: Exploration Beyond the Boundary of Explored Regions | Abstract
• Other
  • Strategies for Missing Data (a stats reference) - related to our guest lecture from last week

Class Meetings

Monday

• Review of neural network architectures:
  • Wiring doesn’t change: Feed-forward (MLP)
  • Current sample wired to previous sample:
    • Recurrent Networks (Elman; LSTM and GRU)
  • Current sample wired to surrounding samples: Convolutional Networks (CNN)
  • Wiring computed dynamically based on “self-attention”: Transformer
• Tricks
  • Residual Connections
  • Dropout

Wednesday

• finish topics from Monday

Friday

• Reinforcement Learning (learning from feedback)

Contents

• Discussion 12: A State-of-the-Art Language Model (due Wed Apr 13)
• Homework 12 (due Thu Apr 14)

Due this Week