Neural Computation
Today’s ML systems are a mashup of two kinds of computational objects: the traditional sequential programming that we’re used to is still usually the “outer loop” of an ML system, but that code is the caretaker for a very different kind of animal: a highly parallel vector computer controlled by billions of parameters. This pillar is about understanding how that parallel vector computer works and how we can control it.
- How can we represent text, images, and other data as sequences?
- How can we process and generate sequences using neural nets?
- How can models capture and use nuanced long-range relationships?
ML Systems
- How do we evaluate language models?
- Can I run an LLM on my laptop? Can I train one?
- How do I get good-quality results from an LLM?
- How can I use an LLM to make a (semi-)autonomous agent?
Learning Machines
- How can we learn without labeled data? (self-supervised learning)
- How do foundation models learn generalizable patterns from massive datasets?
- How can generative agents learn to improve their behavior from feedback?
- Some current models can learn at test time (e.g., in-context learning); how does this work?
Context and Implications
- What problems can we use AI to solve?
- What should we use AI for?
- What are the limits of AI systems? Is superhuman AI imminent?
- What might happen socially when AI systems are deployed broadly? (effects on work, education, creativity, …)
- How might we design AI systems to align with human values? to honor each other and our neighbors? What are the risks if we don’t?
- How do privacy and copyright relate with AI? Is generative AI all theft?
- What is creativity? Agency? Truth?