Objectives addressed:
- [MS-LLM-Tokenization] I can explain the purpose, inputs, and outputs of tokenization.
- [MS-LLM-API] I can apply industry-standard APIs to work with pretrained language models (LLMs) and generative AI systems.
- [LM-SelfSupervised] I can explain how self-supervised learning can be used to train foundation models on massive datasets without labeled data.
Work through this notebook today to learn about what the outputs of a language model look like. You’ll see how it’s a token-by-token classification model.
The main objective is for us to understand the output of a language model. We’ll see that the output is a probability distribution over the vocabulary for each token in the sequence.
Logits in Causal Language Models
(name: u09n1-lm-logits.ipynb; show preview,
open in Colab)